Programmer's Guide to cereal

Miloslav Trmač

You can redistribute and/or modify this program under the terms of the GNU General Public License version 2 as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place Suite 330, Boston, MA 02111-1307, USA.


Table of Contents

1. Introduction
2. General Conventions
3. cereal Modules
4. KDE UI extensions
5. cereal Front-end Interface
Module Handling
Port Space Handling
Port Handling
Port Connection Handling
Expression Handling
Breakpoint Handling
Initialization, Finalization, Saving and Loading
Starting the Emulation
8051-specific Interfaces
6. Happy Hacking!

Chapter 1. Introduction

The main strength of cereal, its ability to emulate connected sets of devices, can rarely be fully exploited without writing custom modules emulating devices specific to your application. This document aims to provide enough information to get you started writing these modules and associated KDE UI. The last chapter describes interfaces you'll need for writing a new front-end to the emulation engine.

This document assumes reasonable user's knowledge of cereal (do read the complete User's Tutorial), knowledge of the C programming language in its latest standardized version known as C99, and ability to use libxml2 for saving and loading data (which takes about fifteen minutes to learn). For writing KDE UI extensions, ability to program KDE programs is obviously essential.

This document is only meant to be an overview, assuming you can find the actual interface definitions in the header files. You are also encouraged to use the provided modules as examples (remember, grep is your friend).

Additionally, in the sample directory you can find two modules which were used in first real usage of cereal. While they can be viewed as an example how not to do things (namely combining UI and emulation code), they are included also to show how easy it can be.

Chapter 2. General Conventions

Errors are reported by returning a value described in the function synopsis. The responsibility to inform the user lies with the first function encountering the error condition, therefore if you call a function and receive the error code, you can just pass it to your caller without telling the user again. Errors are reported by calling the error () function, which is provided by the front-end.

Note

The KDE front-ends (cereal_kde and cereal_khwconf) sometimes (quite often actually) need to try whether a particular operation is available or not without displaying any error message, or displaying it in less obtrusive way (e.g. in the Evaluate/Modify dialog box). This is achieved by indirecting through error_handler, which can be locally overridden to redirect the error messages.

For memory allocation in C use the provided xmalloc (), xrealloc (), xstrdup () and xxmlMalloc (), unless you are prepared to handle out-of-memory conditions gracefully. The provided functions check the result and abort when the allocation fails.

For reading unsigned integers from strings, you may want to use the get_num () function which is simpler to use than strtoul () and unlike strtoul () also checks that there are no trailing characters left.

Chapter 3. cereal Modules

To implement a cereal module, you need to create a cereal_module structure which describes the module. To create a built-in module, you need to add your module to cereal linking process and add it to module_list.h. This will cause the module to be automatically registered on cereal startup. To create a dynamically-loaded one, create a shared library with a function struct cereal_module *register_self (void), which calls cereal_module_register () for your cereal_module and returns a pointer to it. This library needs to be named libcerealfoo.so (where foo is any string) and placed in a directory given by the CEREAL_MODULE_DIR environment variable.

The cereal_module structure contains a few function pointers and description of ports the module provides. You can view the cereal_module as an object with virtual methods—except it is done in C. The following methods have to be defined:

NameDescription
mi_newCalled by the back-end to create a new instance of your module. This instance is by convention represented in a structure called mi (for Module Instance) and is completely private to your implementation. After creating the instance, return a pointer to it. In the unlikely event you don't need to keep any state, allocate a dummy non-null pointer using malloc (1) (note that malloc (0) may return NULL).
mi_deleteCalled by the back-end to destroy an instance created by a previous call to mi_new. Perform needed cleanups and free the data.
set_optionCalled to set an option of the given instance. Options can be changed in the cereal_khwconf module properties dialog. The set of options is your choice. If you need to pass complicated commands to your module which can not be reasonably implemented using ports, you can define options that are hidden from the user and used by your front-end to communicate with the emulated module. This is used in the 8051 module to allow the front-end to load an Intel HEX file to the internal program memory.
get_optionThe obvious counterpart of set_option.
save_setupCalled to save the current setup to an XML file. The setup means properties of your object that don't change during emulation. This usually means only the options. Don't save the port connections here, this is done automatically by the back-end. Saving and loading data in XML is simple (and boring) work, see 8051/8051.c for a largish example. You should select an XML namespace for your module and save all data not defined in the DTDs in doc/dtd using this namespace.
load_setupThe counterpart of save_setup. If your save_setup function just saves the options in a format <ns:option>value</ns:option>, you can use the generic_module_load_setup_options () function instead of parsing the XML on your own. Other helpers to note include get_xml_num () and get_xml_enum ().
save_stateCalled to save current emulation state. This does not include the setup, the back-ends saves both in a single file by calling both save_setup and save_state. For saving scheduling events (see below), use schedule_save_event ().
load_stateThe counterpart of load_state. For loading scheduling events use schedule_load_state ().

It may make sense for set_option, get_option, load_setup and save_setup to do nothing. In that case, you can initialize the cereal_module members using generic_module_… functions (you can't just leave the pointers NULL).

You should already know that modules communicate with each other using ports and ports are grouped in spaces. Ports in a space should be logically grouped, and they all must have the same width,the number of bits that are transfered in one operation. The cereal_module structure contains a module_width structure for each allowed port width (which is internally represented by enum port_width, which currently allows 1, 8 and 16 bits). Thus any given port is completely identified by the module, width, space (zero-based index) and port (zero-based index).

Note

It hasn't been explicitly stated yet, so here we go: cereal can only represent digital information with a simple, definite value. It can't directly represent analog computers or undefined states when the logical value is changing.

In the module_width structure, you need to fill in a pointer to array of structures describing individual port spaces with given width, and number of entries in this array. These entries are of type struct space and contain just pointers to functions. First of them is a get_size function, which returns number of ports in this space. Usually, this will be a constant, but it may also depend on a module option (such as the data_mem_size option of the 8051 module). The size of the space may not change during emulation (it must depend only on the module setup).

The space structure contains also two sets of function pointers, one per each port type (read, write, display, modify). The get_fn[] pointers are used to get functions used to access your ports (e.g. get_fn[PORT_TYPE_READ] is used to get a function that returns the current value of the port, to be displayed to the user). The set_fn[] pointers are used to connect other modules: if you connect for writing port A/A/A to port B/B/B (in that order), the get_fn[PORT_TYPE_WRITE] functions is called for the port B/B/B and the result is passed to the set_fn[PORT_TYPE_WRITE] function for port A/A/A. As a result, whenever port A/A/A has a value to write, it will call the function, which will handle the write on behalf of port B/B/B.

The functions are represented as struct port_fn, which contain the needed function pointer together with the destination module instance pointer and a function-specific data. There are two reasons a pure function pointer is not enough: First, you'll usually want to use a common function for the whole address space (i.e. when the address space represents bytes stored in a memory chip), so you need to preserve the port number given to the get_fn[] function. To understand why the instance pointer is needed, recall what is the function of sfr_ext space in the 8051 module. The get_fn[] of 8051 sfr space is implemented by returning the function connected (via set_fn[]) to the corresponding port of the sfr_ext space. Thus the following accesses (which really transfer data) go directly to the module connected on sfr_ext port without passing through the 8051 sfr—sfr_ext combination on each access.

The get_fn[] and set_fn[] pointers are in some ways different from the other function pointers used in cereal. First, the pointers can be NULL, which means that the operation is not supported. For a port to make sense, you need to support at least one type of access, though (unless the port serves as a placeholder to keep a relation between port addresses and some externally given addressing scheme (8051 SFR addresses)). Another difference is that the set_fn[] functions just return error code without telling the user.

In your module, you need to implement the “action” functions returned by get_fn[] and store port_fn structures for the ports that you need to connect to. In the set[] fn functions you then return either 0 if OK, ENOTSUP if the particular port doesn't support this type of access, or EBUSY if the port is already connected.

Now that you know how to create a module and how to make it communicate, it is time to make it do some work. Although some modules (the simple bit_report and bit_value modules) can work with just immediately handling read/write/modify/display requests, your module will quite often need to simulate a separate process running in time (e.g. the uart module sending data). This is done by representing the process you emulate as a state machine, which can react to port accesses and timer expirations.

The timer expirations are represented by creating scheduling events. Schedule event is a timer that can be armed to trigger at a specified time. Each time the timer triggers, a handler function associated with the event is called. You can allocate scheduling events in your mi_new function by calling schedule_new () and delete it in your mi_delete function by calling schedule_delete (). To “arm” the event, call schedule_add (), specifying the handling function and relative time after which the event triggers. At any time you can cancel the armed event by calling schedule_cancel (). Usually, you'll want to arm one or more events in the mi_new function to start your process (in fact, cereal needs at least one such module to do anything at all), but you may also want to arm an event in response to writing to one of your ports.

The back-end keeps the “emulation time” in the cereal_time variable, which you can copy any time to get a “snapshot”. At a later time, you can call schedule_difftime () to measure time between the snapshot and current time (this can be useful for example for measuring frequency on one of your input ports).

Sometimes, you'll want to report an error or a warning. Use the above-mentioned error (), and also mark the fact by setting the bit ER_ERROR or ER_WARNING in the variable emulation_result. The front-ends should stop emulating when an error occurs, and they look at the emulation_result to do so. For similar reasons, if you are emulating a CPU and you have finished emulating an instruction (as defined by the usual meaning of the “Step” command), set the ER_INSN bit.

After creating the module, you need to make the vast possibilities available to the user. Create a module_name.xml in the xml directory. See the other files in that directory and doc/dtd/cereal_module.dtd for detailed information.

Chapter 4. KDE UI extensions

KDE UI extensions are implemented as dynamically loaded KParts plugins to KCMainWin, the emulator main window. Using KParts mostly amounts to copying the boilerplate code. You create a class derived from KLibFactory which can instantiate your plugin. This plugin checks that it is really connected to KCMainWin and then plugs its actions to its interface using the XMLGUI mechanism.

Note

When copying the boilerplate code, don't use LDFLAGS=$(KDE_PLUGIN), because the plugin references symbols in the main executable. Instead, use the flags defined by $(KDE_PLUGIN) without the -no-undefined flag.

Usually, you'll want your module to provide an advanced interface to instances of a particular module type. To do this, check whether the module type is available at all by calling cereal_module_find (), which returns a pointer to its cereal_module structure. Then, when the user invokes this interface, you can call KCMainWin::selectModule () to let the user select the particular module instance that should be used. For a trivial example, see KC8051::loadProgram ().

If you want to create a window, you should create it as a child of the widget returned by KCMainWindow::viewParent (). This will automatically insert the window in the MDI framework. Your window should also in most cases inherit KCWindow and implement the abstract functions. This will cause your window to get notifications whenever the emulated state changes (if your window causes change of emulation state, you have to report this by calling KCMainWin::updateViews ()), to be closed when the user loads a different file, and if you implement KCWindowProvider, it will also automatically manage the View menu.

It is nice to the user to save the state of your interface extension to the XML file the emulation state is saved to, so that the user doesn't have to e.g. reenter breakpoints each time he runs cereal. To do this, implement the KCUIStateHandler interface (it is probably simplest to do so in your KParts plugin), its saveState () and loadState () methods, and register the handler by calling KCMainWin::registerStateHandler (). As a convenience, your windows inherited from KCWindow have also a saveState method. Thus if all state you need to save is associated to your KCWindow descendants, you should implement KCWindow::saveState () to save state created with that particular window, leave KCUIStateHandler::saveState () empty and in KCWindow::loadState () look for all window states saved, recreate the windows and restore their state.

You may find useful these cereal-specific widgets: KCLineEdit and KCListView are variants of KLineEdit and KListView with additional signals, KCExprLineEdit is an KLineEdit with a Port... button which allows the user to easily add port references. KCPortEdit is a KLineEdit with a label above which works as an Evaluate/Modify dialog box, except that the expression is fixed and invisible to the user. Finally KCBitEdit is a widget similar to KCPortEdit, except that it is used to modify a particular bit of the expression.

For the “real” work, you can use the interfaces described in the next chapter. When there is no special interface, just invoke the needed functionality directly (e.g. set_option and get_option module methods). The core of cereal is not written in C++ and does not have trivial wrapper functions around every data member.

Chapter 5. cereal Front-end Interface

cereal does not impose a specific structure to your front-end. For most purposes, you can view the cereal core as a library which you can use when you need to.

Module Handling

To create modules, you need to find the module type you want to operate on. This can be done either by searching by for it by name using cereal_module_find (), or browsing cereal_module_list, the list of available module types.

Then you can create a module instance. Internally, module instances are usually passed around using struct me_entry, which contains everything needed to handle a module—the module type, name and instance data. Thus, you can create a module using me_new, delete it using me_delete (), or rename it using me_rename (). Module names are (mostly for simplicity) restricted to C-like identifiers. If you want to check whether a given name is allowed (for example in a validator of an edit control), use me_name_ok ().

Once a module is created, you can obtain a pointer to its me_entry using me_find () (which doesn't report a not-found error to the user) or me_get () (which does). To handle options of the module, call the set_option and get_option functions directly. All module instances can be enumerated by browsing the me_list.

Port Space Handling

Port spaces are mostly a grouping of ports by a common set of get_fn[] and set_fn[] functions, so there is no need to care about them for the sole purpose of emulating. However, spaces are better presented to the user using human-readable names instead of a tuple (width, index). The names are defined in the module XML file and functions space_create_name () and space_find_name () allow you to directly transform space names and indexes.

Note that these functions (and whole cereal core) use for representing port widths enum port_width, not the integral values. Therefore, use WIDTH_8 instead of just 8. To communicate with the user, you can convert widths between these formats using port_width_value array and get_port_width () function.

Port Handling

Ports themselves are more interesting than port spaces. A port of a given width can be identified by its module instance and its space and port indexes, which are grouped in struct port_id.

Similarly to spaces, ports can be also named in the module XML file. Use port_create_name () and port_find_name () to convert names and indexes. On a slightly higher level, to convert a port name in the cereal expression format of module/space/port, use port_find.

Once you have a determined a port, you can read its value (using the “display” function, without side-effects) using port_disp_W () and set it (using the “modify” function) using port_set_W (), where W is 1, 8 or 16, depending on the port width. Usually it is easier to use cereal expressions interface described later.

When communicating with the user, you'll want to present human-readable names for the port types, which are internally represented by the PORT_TYPE_* values. To do this, use the port_type_name array and port_type_find () function.

Port Connection Handling

A port connection is represented by a struct connect_entry, which contains the connection type, width and the two ports. All current connections are in connect_list. To connect two ports, fill in a “template” and pass it to port_connect (), to disconnect them, use port_disconnect ().

When creating a GUI for connection modification, it might be undesirable that modifying a connection (by disconnecting it and creating a new one) may change address of its connect_entry. In that case, disconnect the connection using port_do_disconnect (), modify its members and reconnect it using port_do_connect (). Also when creating the GUI, you'll want to allow the users to cut the time spent creating the connections in half by allowing them to create, modify and delete the connections in read/write pairs. When you have one connection and want to find the other in its read/write pair (if it exists), call port_peer_connection ().

Expression Handling

To evaluate a cereal expression, use expr_eval (), to modify one expression to a value of another (as in the Evaluate/Modify dialog), use expr_modify. If you are often evaluating the same expression over and over, you should compile it by calling expr_compile () which compiles the expression to a P-code, and then just pass the P-code to expr_exec (). Evaluating compiled expression was about five times faster than parsing the expression each time for the expressions I have tested.

Breakpoint Handling

Breakpoint is represented by a struct breakpoint, which contains the expression both in text and compiled form and a place for front-end-specific data, (such as a breakpoint number in a gdb-like interface). Create a breakpoint using breakpoint_new (), delete it using breakpoint_delete (), change it in-place using breakpoint_change ().

To evaluate a breakpoint, you can use expr_eval () directly, or use breakpoint_eval (), which will return just 0 or 1 (error in evaluation is taken as 0). All breakpoints are in breakpoint_list, to check whether some of them triggered use breakpoint_check () (which also handles the “triggers only when was zero before” semantics, unlike breakpoint_eval ()).

Initialization, Finalization, Saving and Loading

The first thing you do with the cereal core should be calling setup_init () (which registers all available module types) and the last thing you do should be calling setup_destroy (), freeing used memory.

To save and load the "setup" (as in cereal_khwconf), use setup_load () and setup_save (). To save and load current emulation state (together with the used setup), use state_load () and state_save (). The two latter functions also allow you to save front-end-specific state along with the emulation state. Note however, that this front-end-specific state should not be needed for emulation, because it can be lost when the emulation state is opened in another front-end.

Starting the Emulation

Emulation is done by handling armed scheduling events in chronological order. To do so, call schedule_run (), which will handle the first event in the queue. If there are no events, it returns ER_HALT. You may want to check for this condition in advance using schedule_pending ().

In case you wonder, yes, this section has been placed almost at the absolute end intentionally. I wanted you to at least skim through all the interfaces, you'll need most of them anyway.

8051-specific Interfaces

If it is customary in your CPU architecture to store programs in the Intel hex format, use load_hex_file (), load_hex_stdio (), load_xml_stdio () and save_hex_xml () functions.

The 8051 disassembler used in cereal_disasm and the 8051 KDE UI extension is available as i8051_disasm ().

Chapter 6. Happy Hacking!