Java quick

Here we introduce Alida's operator concept with some code snippets of ALDOperator and sub-classed demo operators. we focus on Alida's capabilities to automatically generate user interfaces.

First operator

As a first example of an Alida operator we implement the row or column wise sum for a 2D array of Doubles. The class MatrixSum extending ALDOperator features three member variables holding the input 2D array, an enum to indicate the mode of summation (ROW or COLUMN), and an 1D array of the sums to be computed and returned by the operator. Alida requires only to annotate these members with the @Parameter annotation which declares

• the direction (IN, OUT, INOUT),
• whether the parameter is required,
• an optional textual description,
• and a label used, e.g. in the graphical user interface automatically generated

to execute the operator.

It is important to add a public standard constructor (without arguments) to be able to use Java's reflection mechanism. Finally, the abstract method operate() of ALDOperator has to be overridden implementing the functionality of the operator.

@ALDAOperator(genericExecutionMode=ALDAOperator.ExecutionMode.ALL,
				level=ALDAOperator.Level.APPLICATION)
public class MatrixSum extends ALDOperator {

  /** Choose row or colum wise sum
    */
  public static enum SummarizeMode {
	/** row wise */
	ROW,

	/** column wise */
	COLUMN
  }

  /**
   * Input matrix
   */
  @Parameter( label= "Input matrix", required = true, 
  		direction = Parameter.Direction.IN, description = "Input matrix.")
  private Double[][] matrix;

  /**
   * Mode of summarizing
   */
  @Parameter( label= "Summarize mode", required = true, 
  		direction = Parameter.Direction.IN, description = "Sum over columns or rows?")
  private SummarizeMode summarizeMode = SummarizeMode.ROW;

  /**
   * 1D Array of sums.
   */
  @Parameter( label= "sums",  
  		direction = Parameter.Direction.OUT, description = "Row or column wise sums.")
  private Double[] sums = null;

	/**
	 * Default constructor.
	 * @throws ALDOperatorException
	 */
	public MatrixSum() throws ALDOperatorException {
	}

	/**
	 * Constructor.
	 * 
	 * @param matrix	Input matrix.
	 * @throws ALDOperatorException
	 */
	public MatrixSum(Double[] [] matrix) throws ALDOperatorException {
		this.matrix = matrix;
	}

	@Override
	protected void operate() {
		if ( matrix == null ) 
			sums = null;

		// calculate sums
		if ( summarizeMode == SummarizeMode.ROW ) {
			sums = new Double[matrix.length];
			for ( int row = 0 ; row < matrix.length ; row++ ) {
				sums[row] = 0.0;
				for ( int col = 0 ; col < matrix[0].length ; col++ )
					sums[row] += matrix[row][col];
			}
		} else {
			sums = new Double[matrix[0].length];
			for ( int col = 0 ; col < matrix[0].length ; col++ ) {
				sums[col] = 0.0;
				for ( int row = 0 ; row < matrix.length ; row++ )
					sums[col] += matrix[row][col];
			}
		}
	}

These are the basic requirements for the operator to be used on the programming level. An example of this use is included in the example in the next section.

If we further annotate the class with

@ALDAOperator(genericExecutionMode=ALDAOperator.ExecutionMode.ALL)

this is all needed to also facilitate execution of this operator via a graphical and a command line user interface automatically generated by Alida. (Setting level=ALDAOperator.Level.APPLICATION declares this operator an application which is used in the GUI to control display of available operators.)

Invocation via a graphical user interface

Alida comes with one single application to execute Alida operators with a automatically generated graphical user interface which may be started from command line by

java de.unihalle.informatik.Alida.tools.ALDOpRunnerGUI

This will pop up a window to choose an operator to execute. Arranged according to the package structure all operators allows to be executed via the graphical user interface according to their genericExecutionMode are displayed. Initially packages are unfolded up to a predefined depth. Unfold the demo package, select MatrixSum, and choose the "Configure Operator" button. This will pop up another window which allows you to configure the input parameters of the operator. Important note: After finishing to input the data matrix entering the final matrix elements you have to select a previous matrix element due to subtle AWT details. For the enumeration to select the mode Alida has automatically generated a combo box to allow convenient selections. If you are finished with the parameter configuration you want to invoke the operator using the run button. On completion of MatrixSum the interface will pop up the result window which allows you to inspect the outcome of the operation.

Invocation via command line

The command line user interface of Alida allows to invoke all Alida operator properly annotated to allows generic execution.

You may invoke the matrix summation operator by

java de.unihalle.informatik.Alida.tools.ALDOpRunner MatrixSum matrix='[[1,2,3],[4,5,6]]' sums=-

which returns as result on standard output

sums = [6.0,15.0]

Parameter values are specified as name=value pairs. Alida's syntax for 2D array should be self-explanatory from this example. As the mode of summation is not supplied as a parameter its default is used

Note, the command

java de.unihalle.informatik.Alida.tools.ALDOpRunner MatrixSum matrix='[[1,2,3],[4,5,6]]' 

will return no output as the command line user interface returns only output parameters requested.

The enumeration defined in MatrixSum is supported by the user interface without further action required as shown in the next example. This also demonstrates redirection of output to a file, sums.out in this case.

java de.unihalle.informatik.Alida.tools.ALDOpRunner MatrixSum matrix='[[1,2,3],[4,5,6]]' 
	summarizeMode=COLUMN sums=@sums.out+

Input can be read from file as well:

java de.unihalle.informatik.Alida.tools.ALDOpRunner MatrixSum matrix=@data sums=-+

where the file data contains the string defining the matrix, e.g., [[1,2,3],[4,5,6]]

Adding more features to an operator

We now generalize this example to realize not only summation over rows or columns, but arbitrary summarizing operations. This shows Alida's feature to allow an operator as parameter of another operator.

Implementation

First we generalize ALDArraySum to the operator ApplyToMatrix which also takes a 2D array and an enum indicating the mode of marginalization (ROW or COLUMN). It takes an additional input parameter which specifies the operation to be applied on each row or column.

This parameter is itself an Alida operator and of type ALDSummarizeArrayOp which is implemented as an abstract class. This abstract operator defines a summarizing operator which takes a 1D array as input and returns a summarizing scalar. As this is an abstract class there is no need to override the operate() method, however some getter and setter methods are provided.

  @Parameter( label= "Input 1D array", required = true, 
  		direction = Parameter.Direction.IN, description = "Input array (1D).")
  protected Double[] data;

  /**
   * Summarizing scalar
   */
  @Parameter( label= "Summarizing scalar",  
  		direction = Parameter.Direction.OUT, description = "Summarizing scalar of the 1D arra")
  protected Double summary = null;

	/**
	 * Default constructor.
	 * @throws ALDOperatorException
	 */
	public ALDSummarizeArrayOp() throws ALDOperatorException {
	}

	/**
	 * Returns the 1D array
	 * @return data array
	 */
	public Double[] getData() {
		return this.data;
	}

	/**
	 * Sets the 1D array
	 * @param data
	 */
	public void setData( Double[] data) {
		this.data = data;
	}

Now we add concrete examples of such a summarizing operation, in this case summation (ALDArraySum), to return the mean (ALDArrayMean), and the minimum (ALDArrayMin). Each implements the operate() method and has to supply a standard constructor. In this example we add another constructor for convenience. This operators are declared as operators on the standard in contrast to application level, as they are not expected to be invoked as an application. However, setting the level to standard in the menu of the graphical user interface stills allows their execution. When extending the abstract super class it is necessary to annotate the class with @ALDDerivedClass in order to allow Alida's dataIO mechanism to find the derived class in the automatically generated user interface. This holds for other parameter types as well. More specifically, if an instance of a class is to be supplied in an automatically generated user interface as a value for a parameter of one of its super classes, Alida requires the annotation @ALDDerivedClass.

@ALDDerivedClass
@ALDAOperator(genericExecutionMode=ALDAOperator.ExecutionMode.ALL,
        level=ALDAOperator.Level.STANDARD)
public class ALDArraySum extends ALDSummarizeArrayOp {

    @Override
    protected void operate() {
        summary = 0.0;
        for ( int i = 0 ; i < data.length ; i++ )
            summary += data[i];
    }

    /**
     * Default constructor.
     * @throws ALDOperatorException
     */
    public ALDArraySum() throws ALDOperatorException {
    }

Now we are ready to implement the ApplyToMatrix operator, which also demonstrates supplemental parameters. This supplementals, e.g., control debugging output or returning of intermediate results. For demo purposes we declare a supplemental input parameter returnElapsedTime. If it is set to true the operator will return the elapsed time in a second supplemental parameter with direction output.

Again, the operation is implemented in the operate() method and the remainder of the class supplies getter and setter methods for convenience. The operate() method give also an example of the invocation of an operator on the programming level. In this case, an instance of the operator is already passed as a parameter. Its parameters are set, in this case each 1D array to be summarized in turn. Upon return from the method runOp() the results may be retrieved from the operator object, in this example with the getSummary() method. Besides getter and setter methods as implemented in each operator Alida provides also a generic get and set methods applicable to all parameters of an operator. Note, that the operator is not invoked by its operate() method, but via the runOp() method implemented the base class ALDOperator. This methods validates the parameters before invocation of operate(). Furthermore, it take all necessary measures for Alida's processing history which automatically logs all manipulative actions on the data and corresponding parameter settings.

@ALDAOperator(genericExecutionMode=ALDAOperator.ExecutionMode.ALL,
            level=ALDAOperator.Level.APPLICATION)
public class ApplyToMatrix extends ALDOperator {

    /** Choose row or colum wise sum
      */
    public static enum SummarizeMode {
      /** row wise */
      ROW,
      /** column wise */
      COLUMN
    }

    /**
     * Input matrix
     */
    @Parameter( label= "Input matrix", required = true, 
          direction = Parameter.Direction.IN, description = "Input matrix.")
    private Double[][] matrix;

    /**
     * Mode of summarizing
     */
    @Parameter( label= "Summarize mode", required = true, 
          direction = Parameter.Direction.IN, description = "Sum over columns or rows.")
    private SummarizeMode summarizeMode = SummarizeMode.ROW;

    /**
     * Summarizing opererator
     */
    @Parameter( label= "Summarizing operator", required = true, 
          direction = Parameter.Direction.IN, description = "Specifies the summarizing operation to apply")
    private ALDSummarizeArrayOp summarizeOp;

    /**
     * 1D Array of summaries.
     */
    @Parameter( label= "summaries",  
          direction = Parameter.Direction.OUT, description = "Row or column wise summaries")
    private Double[] summaries = null;

    /**
     * Supplemental to request elapsed time to be returned
     */
    @Parameter( label= "Return elapsed time", 
          direction = Parameter.Direction.IN, description = "Request elapsed time consumed to be returned",
        supplemental=true)
    private boolean returnElapsedTime = false;

    /**
     * Elpased time 
     */
    @Parameter( label= "Elapsed time",  
          direction = Parameter.Direction.OUT, description = "Elapsed time of operation in milliseconds",
        supplemental=true)
    private long elapsedTime;

    /**
     * Default constructor.
     * @throws ALDOperatorException
     */
    public ApplyToMatrix() throws ALDOperatorException {
    }

    /**
     * Constructor.
     * 
     * @param matrix    Input matrix.
     * @throws ALDOperatorException
     */
    public ApplyToMatrix(Double[] [] matrix) throws ALDOperatorException {
        this.matrix = matrix;
    }

    @Override
    protected void operate() throws ALDOperatorException,ALDProcessingDAGException {
        if ( returnElapsedTime ) 
            elapsedTime = System.currentTimeMillis();

        if ( matrix == null ) 
            summaries = null;

        // calculate summaries
        if ( summarizeMode == SummarizeMode.ROW ) {
            summaries = new Double[matrix.length];
            for ( int row = 0 ; row < matrix.length ; row++ ) {
                summarizeOp.setData(matrix[row]);
                summarizeOp.runOp();
                summaries[row] = summarizeOp.getSummary();
            }
        } else {
            summaries = new Double[matrix[0].length];
            Double[] tmp = new Double[matrix.length];
            for ( int col = 0 ; col < matrix[0].length ; col++ ) {
                for ( int row = 0 ; row < matrix.length ; row++ )
                    tmp[row] = matrix[row][col];

                summarizeOp.setData(tmp);
                summarizeOp.runOp();
                summaries[col] = summarizeOp.getSummary();
            }
        }

        if ( returnElapsedTime ) 
            elapsedTime = System.currentTimeMillis() - elapsedTime;
    }
    
    // ==============================================================
    // Getter and setter methods
    /** Get value of returnElapsedTime.
      * Explanation: Request elapsed time consumed to be returned.
      * @return value of returnElapsedTime
      */
    public boolean getReturnElapsedTime(){
        return returnElapsedTime;
    }

    /** Set value of returnElapsedTime.
      * Explanation: Request elapsed time consumed to be returned.
      * @param value New value of returnElapsedTime
      */
    public void setReturnElapsedTime( boolean value){
        this.returnElapsedTime = value;
    }

Invocation via a graphical user interface

If the graphical interface is still running just select our new operator and begin to configure it. For the parameter summarizing operator you have a choice of all operators extending the abstract operator ALDSummarizeArrayOp. All which is necessary on the implementation side is proper annotation of the extending classes with @ALDDerivedClass. As the selected operator may have its own parameters you may want to configure it. In our example this is not necessary as the input array is, of course, supplied by the ApplyToMatrix operator. Do not forget to input your data before hitting the run button. After return, again a result window give you the results of the operation. Note, if you did not tick Return elapsed time" this window will show zero for the time elapsed as the operator has not been request to stop the time.

Invocation via command line

When invoking the ApplyToMatrix operator from command line we have to handle derived classes as value for parameters. In the graphical user interface Alida features a combo box where we may choose from. In the command line interface Alida allows to prefix the value of a parameter with a derived class to be passed to the operator. This is necessary as Alida as, of course, no way to itself decide if and which derived class is to be used. Alida's syntax is to enclose the class name in a dollar sign and a colon. As evident in the following example, abbreviations are of the fully qualified class name are accepted as long as they are unambiguous.

java de.unihalle.informatik.Alida.tools.ALDOpRunner Apply \
	matrix='[[1,2,3],[4,5,6]]' \
	summarizeMode=ROW \
	summarizeOp='<math>ALDArrayMean:{}' \
	summaries=-

results in

summaries = [2.0,5.0]

ALDOpRunner may be persuaded to show all operators derived from ALDSummarizeArrayOp and known within the user interface if we enter an invalid class name:

java de.unihalle.informatik.Alida.tools.ALDOpRunner \
	Apply matrix='[[1,2,3],[4,5,6]]' \
	summarizeMode=ROW summarizeOp='</math>dd:{}' \
	summaries=-

yields

ALDStandardizedDataIOCmdline::readData found 0 derived classes matching <dd>
      derived classes available:
	de.unihalle.informatik.Alida.demo.ALDArrayMean
	de.unihalle.informatik.Alida.demo.ALDArrayMin
	de.unihalle.informatik.Alida.demo.ALDArraySum
ERROR: reading parameter <summarizeOp> returns null

Supplemental parameters are handled like other parameters

java de.unihalle.informatik.Alida.tools.ALDOpRunner Apply \
	matrix='[[1,2,3],[4,5,6]]' \
	summarizeMode=COLUMN \
	summarizeOp='<math>ALDArrayMin:{}' \
	summaries=- \
	returnElapsedTime=true \
	elapsedTime=-

gives

	summaries = [1.0,2.0,3.0]
	elapsedTime = 4

Adding more data types as parameters

Alida provides automatic IO of primitive data types, enumerations, arrays, collections, and operators. In addition so called parameterized classes are supported. Any Java class may be declared to be a parameterized class in Alida by annotating the class @ALDParametrizedClass as shown in the class ExperimentalData1D. All member variables to be known to and handled by Alida's user interface simply need to be annotated with @ALDClassParameter.

Here we implement a toy version of experimental data ExperimentalData1D. A complete experiment contains data of a time series of measurements of variable length. Additional information is a descriptive string, the time resolution in milliseconds, and whether baseline correction has been applied.

The class is annotated by @ALDParametrizedClass, and and all members to be handle in Alida'a user interfaces are to be annotated with @ALDParametrizedClass. The label field has the same semantics as for parameters of operators. These annotations are the only implementational overhead to allow Alida to automatically generate user interfaces where the parameterized class acts as a parameter.

@ALDParametrizedClass
public class ExperimentalData1D extends ALDData {
	
    /** Description */
    @ALDClassParameter(label="description", dataIOOrder = 1)
    private String description = null;

    /** The data  */
    @ALDClassParameter(label="data", dataIOOrder = 2)
    private Double[] data = null;

    /** Are the data baseline corrected? */
    @ALDClassParameter(label="Baseline corrected",
    			dataIOOrder = 3)
    private boolean baselineCorrected = false;
    
    @ALDClassParameter(label="Time resolution in milliseconds", dataIOOrder = 4)
    private Float timeResolution = Float.NaN;

    /** 
     * Standard constructor is required
      */
    public ExperimentalData1D() {
    }
    
    /** Constructor for an experiment.
      * Baseline correction is assumed to be false and nothung known about
      * the time resolution.
      *
      * @param  description   a textual description of the experiment
      * @param  data   measurements
      */
    public ExperimentalData1D( String description, Double[] data) {    
        this( description, data, false, Float.NaN);
    }

This is shown below for a simple normalizing operator SmoothData1D which takes experimental data as input an returns a new instance of ExperimentalData which contains smoothed data.

@ALDDerivedClass
@ALDAOperator(genericExecutionMode=ALDAOperator.ExecutionMode.ALL,
              level=ALDAOperator.Level.APPLICATION)
public class SmoothData1D extends ALDOperator {

	public enum SmoothingMethod {
		MEDIAN, MEAN, GAUSSIAN
	}

	/** 1D Experiment
	 */
	@Parameter( label= "1D Experiment", required = true, 
			direction = Parameter.Direction.IN, 
			description = "1D Experiment",
			dataIOOrder = 1)
	protected ExperimentalData1D experiment;

	/** Smoothing method
	 */
	@Parameter( label = "Smoothing method", required = true,
			direction = Parameter.Direction.IN,
			callback = "smoothingMethodChanged",
			description = "Smoothing method",
			paramModificationMode = ParameterModificationMode.MODIFIES_INTERFACE,
			dataIOOrder = 2)
	SmoothingMethod smoothingMethod = SmoothingMethod.MEDIAN;

	/** Window width
	 */
	@Parameter( label = "Window width", required = true,
			direction = Parameter.Direction.IN,
			description = "Window width (should be uneven)",
			dataIOOrder = 3)
	Integer width = 3;

	/** Standard deviation of Gaussian
	 */
	@Parameter( label = "Standdard deviation of Gaussian", required = true,
			direction = Parameter.Direction.IN,
			description = "Standdard deviation of Gaussian",
			dataIOOrder = 3)
	Float sigma = 1.0F;

	/** Smoothed 1D Experiment
	 */
	@Parameter( label= "Smothed 1D Experiment",  
			direction = Parameter.Direction.OUT, 
			description = "Smothed1D Experiment",
			dataIOOrder = 1)
	protected ExperimentalData1D smoothedExperiment;


	/**
	 * Default constructor.
	 * @throws ALDOperatorException
	 */
	public SmoothData1D() throws ALDOperatorException {
		// necessary handle dynamic parameters correctly
		smoothingMethodChanged();
	}

	@Override
	protected void operate() {
		this.fireOperatorExecutionProgressEvent(
				new ALDOperatorExecutionProgressEvent(this, 
						"Starting to smooth 1D Data..."));

		Double[] smoothedData;
		if ( smoothingMethod == SmoothingMethod.MEDIAN) {
			smoothedData = median( experiment.getData(), this.width);
		} else {
			smoothedData = smoothByConvolution();
		}

		smoothedExperiment = new ExperimentalData1D( experiment.getDescription() + " (smoothed)", 
				smoothedData, experiment.isBaselineCorrected(), experiment.getTimeResolution());
        }

This mechanism applies in a recursive fashion, i.e. a parameterized class may (recursively) contain a member variable which itself is a parametrized class. Likewise, an operator acting as a parameter of another operator may in turn have a parameter of type ALDOperator.

Invocation via a graphical user interface

Invoking and configuring SmoothData1D from the graphical user interface shows as the only required parameter the experimental data. This parameterized class can be configured in a separate window very similar to to configuration of operators. Likewise the resulting normalized experimental data may be inspected in their own window.

Obviously this is a toy example, as we would not expect the measurements to be entered manually, but rather stored and read from file in a specialized format. This is one of the rare cases where custom data IO provider need to be implemented, in this case for ExperimentalData1D.

Invocation via command line

Again, invocation from command line is provided by Alida in an automatic way with no further implementational overhead. The syntax for parameterized classes es a comma separated list of name=value pairs enclosed in curly brackets where name refers to annotated member variables of the parameterized class.

java de.unihalle.informatik.Alida.tools.ALDOpRunner SmoothData1D \
	experiment='{ baselineCorrected=false , \
                      description="my experiment" , \
                      data=[1.0,2.0,2.2,3.3,2.0,1.0,1.0,1.0,1.0,2.0,3.3,2.0] }' \
        smoothingMethod=GAUSSIAN \
	smoothedExperiment=-

yields

smoothedExperiment = { baselineCorrected=false , 
    data=[1.28,1.85,2.37,2.76,2.05,1.23,1.01,1.01,1.23,2.05,2.73,2.35] , 
    timeResolution=NaN , 
    description="my experiment" (smoothed) }

If a class derived from ExperimentalData1D was to be supplied to the operator, the curly brackets can be prefixed by a derive class definition starting with a dollar sign and ending with a colon as shown for the summarizing operators above.

Graphical workflow editor: Grappa

Most of the time complex data analysis tasks cannot be solved by only applying a single operator to the data. Rather, selections of various operators need to be combined into more sophisticated {\em workflows} to extract desired result data. Alida inherently supports the development of such workflows. Grappa, the Graphical Programming Editor for Alida allows for designing and manipulating workflows via graph edit operations, hence, offers an intuitive interface and large flexibility for developing workflows.

A workflow in Alida is defined as a graph data structure. Each node of the graph represents an Alida operator, while edges between different nodes encode the flow of data and control. Each node owns a selection of input and output ports which are associated with the operator's parameters. Consequently, edges are directed, i.e., an edge always connects an output port of one operator node with an input port of another. Grappa visualizes such workflow graphs and supports manual editing, manipulation, and also workflow execution and analysis of results.