Next Contents

Chapter 1:   Introductory Remarks:

Our style in this report is to use the ``royal'' we rather than any sort of third person narrative. We have always felt that this should be interpreted as the author asking the reader to explore the material with the author. To this end, the text is liberally sprinkled with you to encourage this point of view. Please take an active role!

1.1   Copyright Stuff:

We want to provide tools for the neurocontrol community, and to do that the full source and documentation of these tools must be available for use. For good or bad, all of the code described herein has been written by the author, and if you want to use it, modify it or whatever, you should give due credit. Please tell us about errors, design issues and so forth. It will be fun to interact on this material. Send all comments to

petersj@ces.clemson.edu

1.2   FTP Access:

The full source code uses files from folders utility, Xheaders, usecmac and cmacclass. All of these folders were copied as subfolders to the new folder cmacclass_ftp. A listing of this folder then gives:

albert% cd cmacclass_ftp
/home/peterson/programs/cmacclass_ftp
albert% ls
Xheaders/        cmacclass/        usecmac/        utility/
albert%
This code then tarred as follows:

tar cvf - cmacclass_ftp > cmacclass_ftp.tar
a cmacclass_ftp/ 0K
a cmacclass_ftp/cmacclass/ 0K
a cmacclass_ftp/cmacclass/cmaceval.c 2K
a cmacclass_ftp/cmacclass/MakeLibrary 3K
a cmacclass_ftp/cmacclass/echo.c 4K
a cmacclass_ftp/cmacclass/cmac.h 12K
a cmacclass_ftp/cmacclass/func2.c 7K
a cmacclass_ftp/cmacclass/equal.c 2K
a cmacclass_ftp/cmacclass/config.c 4K
a cmacclass_ftp/cmacclass/default_constructor.c 4K
a cmacclass_ftp/cmacclass/func.c 7K
a cmacclass_ftp/cmacclass/getcmac.c 4K
a cmacclass_ftp/cmacclass/cmacinit.c 5K
a cmacclass_ftp/cmacclass/number.c 3K
a cmacclass_ftp/cmacclass/getrms.c 1K
a cmacclass_ftp/cmacclass/copy_constructor.c 2K
a cmacclass_ftp/cmacclass/readte.c 2K
a cmacclass_ftp/cmacclass/readtr.c 2K
a cmacclass_ftp/cmacclass/readw.c 1K
a cmacclass_ftp/cmacclass/constructor.c 1K
a cmacclass_ftp/cmacclass/gettrainsize.c 1K
a cmacclass_ftp/cmacclass/gettestsize.c 1K
a cmacclass_ftp/cmacclass/virtmem.c 9K
a cmacclass_ftp/cmacclass/writew.c 2K
a cmacclass_ftp/cmacclass/cmacfree.c 1K
a cmacclass_ftp/cmacclass/train.c 1K
a cmacclass_ftp/cmacclass/3dgraphics.c 2K
a cmacclass_ftp/cmacclass/cmacfile.c 2K
a cmacclass_ftp/cmacclass/getlevels.c 1K
a cmacclass_ftp/cmacclass/getnumberintervals.c 1K
a cmacclass_ftp/cmacclass/func.fin 2K
a cmacclass_ftp/cmacclass/getminx.c 1K
a cmacclass_ftp/cmacclass/getmaxx.c 1K
a cmacclass_ftp/cmacclass/getwidth.c 1K
a cmacclass_ftp/cmacclass/loadinputsize.c 1K
a cmacclass_ftp/cmacclass/loadoutputsize.c 1K
a cmacclass_ftp/cmacclass/loadtrainsize.c 1K
a cmacclass_ftp/cmacclass/loadtestsize.c 1K
a cmacclass_ftp/cmacclass/getworkingaddress.c 1K
a cmacclass_ftp/cmacclass/getinputsize.c 1K
a cmacclass_ftp/cmacclass/getoutputsize.c 1K
a cmacclass_ftp/utility/ 0K
a cmacclass_ftp/utility/gettxt.c 1K
a cmacclass_ftp/utility/normvec.c 1K
a cmacclass_ftp/utility/MakeLibraryUtility 2K
a cmacclass_ftp/utility/utility.h 1K
a cmacclass_ftp/utility/2d_int.c 1K
a cmacclass_ftp/utility/2d_char.c 1K
a cmacclass_ftp/utility/2d_double.c 1K
a cmacclass_ftp/utility/2d_float.c 1K
a cmacclass_ftp/utility/2d_pointer2int.c 1K
a cmacclass_ftp/usecmac/ 0K
a cmacclass_ftp/usecmac/function/ 0K
a cmacclass_ftp/usecmac/function/func.cfg 1K
a cmacclass_ftp/usecmac/function/func_test.txt 2K
a cmacclass_ftp/usecmac/function/funcinit.txt 1K
a cmacclass_ftp/usecmac/function/graphit.txt 72K
a cmacclass_ftp/usecmac/function/func_train.txt 4K
a cmacclass_ftp/usecmac/function/setfunc.cfg 1K
a cmacclass_ftp/usecmac/function/MakeApplication 2K
a cmacclass_ftp/usecmac/function/simul.c 2K
a cmacclass_ftp/usecmac/function/simul3.c 2K
a cmacclass_ftp/usecmac/function/func.fin 4K
a cmacclass_ftp/usecmac/function/simul.h 1K
a cmacclass_ftp/usecmac/function/simul2.c 2K
a cmacclass_ftp/usecmac/function/MakeApplication3 1K
a cmacclass_ftp/usecmac/function/MakeApplication2 1K
a cmacclass_ftp/usecmac/function/testmodel.c 4K
a cmacclass_ftp/usecmac/function/MakeTest 2K
a cmacclass_ftp/usecmac/function/test_generator.c 3K
a cmacclass_ftp/usecmac/function/MakeGenerator 2K
a cmacclass_ftp/usecmac/function/test1.txt 2K
a cmacclass_ftp/usecmac/function/train1.txt 9K
a cmacclass_ftp/usecmac/function/full1.txt 122K
a cmacclass_ftp/usecmac/function/matlab1.txt 102K
a cmacclass_ftp/usecmac/function/func2.cfg 1K
a cmacclass_ftp/usecmac/function/matlab3.txt 102K
a cmacclass_ftp/usecmac/function/func3.cfg 1K
a cmacclass_ftp/Xheaders/ 0K
a cmacclass_ftp/Xheaders/myconstants.h 1K
The size of the tar file is 586752 blocks. Then, the tar was compressed:

compress cmacclass_ftp.tar
The size of the compressed file is 147205 blocks. The compressed file can be obtained from our web site http://www.math.clemson.edu/ peterson.

We'd like to keep track of how often this material is accessed; please send us a message if you download the source from our web site.

1.3   How to Build the Code:

The compressed file cmacclass_ftp.tar.Z is uncompressed and untarred using the commands:

uncompress cmacclass_ftp.tar.Z
tar -xvf cmacclass_ftp.tar
After this is done, you will have a folder cmacclass_ftp with subfolders usecmac, Xheaders, utility and cmacclass. First, build the utility library libutility.a from the source in the utility folder by typing

make -f MakeLibrary
Then, the source in cmacclass is built by also typing

make -f MakeLibrary
which will generate the libcmac.a library file. You will need to edit these MakeLibrary make files to reflect the proper paths that you are using to store these files. The paths are set up as follows to reflect our needs: 
# -------------------------- #
# paths for this simulation  #
# -------------------------- #
INCLUDE_PATH_UTILITY = /home/peterson/programs/utility
INCLUDE_PATH_CMAC = /home/peterson/programs/cmacclass
INCLUDE_PATH_CONSTANTS = /home/peterson/Xheaders
INCLUDES = -I$(INCLUDE_PATH_UTILITY) -I$(INCLUDE_PATH_CMAC) \
        -I$(INCLUDE_PATH_CONSTANTS)

LIBRARY_PATH_UTILITY = /home/peterson/programs/utility
LIBRARY_PATH_CMAC = /home/peterson/programs/cmacclass
You will need to add your proper paths:

# -------------------------- #
# paths for this simulation  #
# -------------------------- #
INCLUDE_PATH_UTILITY = (YOUR PATH)/utility
INCLUDE_PATH_CMAC = (YOUR PATH)/cmacclass
INCLUDE_PATH_CONSTANTS = (YOUR PATH)/Xheaders
INCLUDES = -I$(INCLUDE_PATH_UTILITY) -I$(INCLUDE_PATH_CMAC) \
        -I$(INCLUDE_PATH_CONSTANTS)

LIBRARY_PATH_UTILITY = (YOUR PATH)/utility
LIBRARY_PATH_CMAC = (YOUR PATH)/cmacclass
Once the libraries are built, you can compile and use the sample application code in the folder usecmac by making appropriate changes in the relevant make files for each application.

1.4   Code Snippets or How to Whet Your Appetite:

In this work, we have laid out a framework on which we intend to build a variety of object--oriented tools to aid our investigations into adaptive real--time neurocontrol. The documentation here consists of both the theoretical development of the Cerebellar Model Articulted Controller (CMAC) neural architecture which is loosely based on the neurophysiology of the cerebellum and concomittant discussion of the object--oriented implementation of this architecture as a class in the language C++. This has been developed in a very general manner and includes a very careful discussion of CMAC convergence.

As mentioned, we have developed C++ classes for the CMAC architecture. Here are some typical code snippets to whet your appetite:

Model Building Using CMACs I:
Given a training and testing set, we can build a CMAC model with code as short as this. Note we include three specialized header files, instantiate the chosen CFFN architecture via the class constructor, initialize the network and train. 
#include "cmac.h"

#define DESIRED_TOL (1.0e-6)
#define STOP_TOL (1.0e-5)
#define RUN_MAX (25)
#define FREQ (1)

void main(int argc,char *argv[])
{
  int run,trainsize,testsize;
  float **intr,**inte,**outtr,**outte,rms,rmstest;

  
  CMAC cmac(argv[1]);                              // set up architecture
  cmac.read_training_data(&intr,&outtr);           //get training data
  cmac.read_testing_data(&inte,&outte);            //get testing data
  cmac.initialize().number();                      //initialize cmac object
  cmac.gettrainsize(&trainsize).gettestsize(&testsize);
  rmstest = cmac.compute_rms(testsize,inte,outte); //find rms on test set
  rms =  cmac.compute_rms(trainsize,intr,outtr);   //find rms on training set
  
  //train the architecture for RUN_MAX iterations
  for(run=0;run<RUN_MAX;++run){
    rmstest = cmac.compute_rms(testsize,inte,outte); //find rms on test set
    rms =  cmac.compute_rms(trainsize,intr,outtr);   //find rms on training set
    if(rms>STOP_TOL) //if rms is too large, keep training
      cmac.train(1,DESIRED_TOL,trainsize,intr,outtr);
    }//run loop 
  printf("rms[%3d] = %12.6e ",run,rms);
  printf("rmstest[%3d] = %12.6e\n",run,rmstest); 
  cmac.write_wts(); //write the weights to a file
}
Note in the above code, we concantenate class function calls as follows:

cmac.gettrainsize(&trainsize).gettestsize(&testsize);
Model Building Using CMACs II:
We can also build a model using vectors of CMAC objects and so forth using instantiations such as

  CMAC T(argv[1]);
  CMAC *cmac;
  cmac = new CMAC[1];
  cmac[0] = T;
or

  CMAC T(argv[1]);
  CMAC cmac; 
  cmac = T;
and there will be no difference in the output of the application.

1.5   The Genesis of This Code:

We first developed CMAC code in C during the summer of 1991 while a faculty fellow at the Johnson Space Center when we were trying very hard to understand the CMAC architecture from the original papers by Albus and Miller. This code was then transformed in 1993 into a stand alone C++ class for use in application and algorithm development in the area of adaptive neurocontrol. This is the code you see here. This C++ code does not use much of the full capabilities of such an object oriented language.

1.6   Candid Assessments of The Code:

There are lots of problems that need to be resolved. We'll just touch on a few.

  1. How many class functions should you have? We can't possibly think of all the class functions that users everywhere need. Our solution so far is to let everyone have the source; then, if a new class function is needed, you write it, recompile the class library and use it in your application code. This question is related to the level of privacy that is implemented in the class. Right now, the CMAC has private elements. The only way a user can use these elements is by writing a class function to access them. Would it be better to make everything public?

    To some extent, this problem is resolved in the neural objects framework by deriving new chold classes as needed.
  2. We have experimented with templates, but their use is very compiler dependent and so at the moment we do not use them in the code. However, we have included a section about our experiences with templates using the SUN PRO C++ compiler.
  3. We also have not implemented exception handling.
  4. An experimental X--Window interface to all of the code has been developed, and that project will soon be ready for public view. There are many interesting design aspects of this kind of interface that we are currently exploring.
  5. We have much of the source code carefully documented, although there is never such a thing as too much explantion! We have also written the user guides and tried to explain as much and as carefully as possible and included the source listings.
  6. The reference list is woefully inadequate. We haven't had time yet to add more or the pertinent literature.
  7. There is a chapter on the underlying neurophysiological model that is as current as we can make it. There is more we could do here, but that will have to wait.

1.7   Future Work:

When you look at this code, you will see many places where the code could have been rewritten using the principles of polymorphism and dynamic binding of computational objects. Indeed, we also have written a stand alone C++ class implementation for Chained Feed Forward Networks and Lagged Chained Feed Forward Networks (all details are in Peterson 44 and we will shortly be realeasing to our web site the source code and documentation for this architecture as well.) With these two explicit examples of stand alone neural architecture classes, we began to try to develop a general software environment based on generic computational modules which we call neural objects. We also wanted to abstract from actual neurobiological data basic principles that would guide us in our software implementation of plastic learning and memory subsystems. This work began in 1993, but we made slow progress until the latter part of 1994. We have now completed a full software design specification for general neural objects (see Peterson 46). This design brings together a large amount of neurobiology, advanced software concepts and mathematics.

In the design document, we attempt to put down on paper our current thoughts on what is to us a fairly ambitious development project for adaptive control. For some time now, we have been interested in developing an environment that allows us to do rapid prototyping of algorithm designs for the control of nonlinear systems using on--line information obtained from sensors in a real--time frame. This requires that we build models of the state and the performance of our system and then use those models to estimate the next time steps control action. This endeavor can be quite mathematical and the implementation of these mathematical ideas into a robust set of software tools is equally daunting. In addition, we have focused on models that are built using function approximation techniques that are loosely based on rather simplistic models of animal neurophysiology--the so--called artificial neural architectures (for example, the CMAC architecture discussed in this volume and the chained FFN architectures discussed in Peterson 44).

We have always believed that we could do much better if we could find the right abstraction of the wealth of neurobiological detail that is available; the right design to guide us in the development of our software environment. Indeed, we feel very strongly that there are three distinct and equally important areas that must be jointly investigated, understood and synthesized for us to make major progress in these areas. We refer to this as the SWH triangle;


Figure 1.1:
we use the term wetware to indicate things that are of neurobiological scope. We use double--edged arrows to indicate that ideas from these disciplines can both enhance and modify ideas from the others. The labels on the edges indicate possible intellectual pathways we can travel upon in our quest for unification and synthesis: ANALOG VLSI is an almost classical chip building strategy which in conjunction with the VHDL layout language provides us with a means to take neurobiological information and implement it on a silicon substrate; hardware and software of the kinds we envision are typically event driven ( and we will have much more to say about that issue); and abstraction is the principle tool that we can use to move back and forth in the two fertile grounds of neurobiology and software.

We have used some of the material in the design document as the basis for a series of theoretical papers on the use of neurobiological structures in the software development. Our design focuses on building software analogs to neurotransmitters, receptors, nerve cell bodies and the post synaptic structure or PSD as the mediator that handles dendritic and axon interactions. In [Peterson 47], we talk specifically about the software modeling of the PSD. The neural objects design is intended to allow us to derive very complicated neural architectures using almost any type of computational module as a ``neuron'' in a connectionist architecture. In [Peterson 45], we discuss carefully how to generate very general back propagation type update strategies for various clusters of neural objects.

In the neural objects framework, each neural architecture is constructed as a derived class from a virtual class of neural objects. The computational strategies that support functions such as training are also classes derived from a core primitive class of computational engines. In [Peterson 48], we discuss how the PSD objectsprovide a reasonable infrastructure for the combining of modular computational objects.

Finally, in the desing document, we spent a lot of time showing the reader how the primitive stand alone classes for the CMAC and Chained FFN previously provide very nice examples of how to decide what a core neural object class should contain. It is well worth going through these discussions. The bottom line is that the implementation presented in this document was a necessary first step in the development of the more general neural object environment (which we are currently in the process of coding in C++.

While the neural objects design was being formulated, we used the CMAC code of this volume as the basis for our first preliminary code that uses dynamically bound computational objects. We modified the stand alone CMAC code to accept as arguments dynamically bound objects that implement various strategies for handling the input space and the resolution of its coarse coding. This code is fully described in [Bolling 6] and the source code will also be available through our web site.

The neural objects framework will enable us to develop fully object oriented versions of many useful computational objects such as:

  1. Chain Feed Forward Class --- this has been done.
  2. Lagged Chain Feed Forward Networks --- this has been done.
  3. Adaptive--Critic Class -- this is under development.
  4. Dynamic Programming Class -- this is under development.
  5. The Multi--grid Class -- this is under development.
  6. Hamilton -- Jacobi -- Bellman Class -- this is under development.
  7. Pontryagin Maximum Principle Class
  8. Principle Component Analysis Class
These tools will all aid in the development of reinforcement learning approaches to real--time learning and control. We have also thought a lot about the neurobiological underpinnings of these architectures and have developed a comprehensive object oriented design for a software environment implementing very general neural object architectures.

1.8   Acknowledgements:

It is a pleasure to acknowledge the support of the NASA Lewis Research Center of Cleveland, Ohio, through grant NAG 3--1311 under the auspices of the Mathematics Support Services Branch with Technical Monitor Jim Pennline. This grant has supported our theoretical and software development for some time now. In addition, it has supported my graduate student Doug Bolling in his M. S. programs during Spring 1994 for software development issues. This has allowed us to make some real progress!

Also, it is with equal pleasure that we acknowledge the beginning of three years of support from the NASA Graduate Research Fellowship Program under the auspices of technical monitor Robert O. Shelton of the Johnson Space Center in Houston, Texas, for my Ph. D. student Linda Lawson.

In addition, we gratefully acknowledge the recent addition of a substantial National Science Foundation Grant ECS--9412430 from the Neuroengineering Program of the Division of Electrical, Communications and Systems Engineering in the Directorate of Engineering with technical monitor Paul Werbos. This grant will provide support for my other graduate student Doug Bolling in his Ph. D. program for an additional three years of development of these ideas as well as other practical and theoretical issues. It is a very exciting time!

I also want to thank my graduate student Christina Juergens for her help in data entry and figure preparation using Adobe Illustrator for the chapter on the neurobiology underlying the CMAC and the chapter on the CMAC primer. NSF funds also supported this effort.

Over the years, we have enjoyed immensely roving intellectual discussions with our friends and colleagues Bob Shelton of the Johnson Space Center, Software Development Branch, and Dan Layne of Martin Marietta, Advanced Development. We're sure they will have a lot to say about this stuff too! We also want to continue to thank Andy Barto for lots of stimulating papers to read. This development effort is in many ways the start of our efforts at understanding his work.

Now for personal stuff: I must thank the unsuspecting students who have sat through a variety of seminars on this material with very little complaint. Some of this material was taught recently in a seminar here at the Mathematical Sciences Department at Clemson University (Spring 1994) and used by two of my Master students in their papers. In particular, I want to thank my cheerful and hard working students Doug Bolling and Linda Lawson (my faithful X and neural architecture explorers) for listening, listening and listening some more..., putting up with my enthusiasm AND using my code in the their work and thereby showing me lots of flaws!

I also want to thank my family (the MIGHTY QUINN, the DANCER QAITLIN and the BEAUTIFUL DOMESTIC ENGINEER and GODDESS PAULETTE) for their support (at least they didn't throw things at me!), their patience (Oh NO! You're not going to work on that AGAIN!) and their love and faith. Thanks and love to you all!

Next Contents