Evaluating Estimation Techniques
Applicability: Cranium, Synapse (core versions 0315+)

One of Cranium's and Synapse's most important capabilities is their ability to select the best estimation technique for a given estimation problem. Both applications perform this selection using techniques' applicability and accuracy rules. These rules determine if an estimation technique is applicable and how accurate it is for a give chemical or mixture. (See documentation on the Model Information Section for additional discussion.)

// Check for infinite dilution - less than 5 wt% if( wpcnt[0] > 0.05 ) return FALSE; // Average error (see reference) SetResult(9.1); // Applicable return TRUE;

Developing applicability and accuracy rules starts with evaluating a technique's estimates against experimental values. This is done using the Technique Evaluation Dialog.

Example: Evaluating an estimation technique
  1. Open the MKS Core Knowledge Base document. (Note that to generate the following documentation, we may have used a different version of the knowledge than you are using. Thus, the property values and results shown below may differ from those you will see while following the example. However, the concepts are the same no matter which knowledge base you use.)
  2. Change to the Techniques Chapter and navigate to the Cp,l (T): Missenard Method [MKS] technique. (See the Navigation Overview documentation for details on navigating chapters and pages.)
  3. Run the Evaluate Technique command from the Commands menu.
  4. The application begins using the technique to estimate the physical properties of every chemical in the current knowledge base. An estimate is generated for every datum in the current document.
  5. Once all estimates have been generated, the results are displayed in the Technique Evaluation dialog. (See here for detailed documentation on the dialog.)

    The evaluation dialog provides numerous statistics for examining the techniques applicability and accuracy. The interactive graphs can also be very helpful.

  6. Select points in the Estimates vs Data graph by clicking and holding down the left mouse button within the graph and then dragging the mouse to enclose the targeted points.
  7. The dialog will compile all points within the selection rectangle and display their corresponding records at the top of the Evaluation Records table.
  8. Note the value for the Avg Abs % Error shown in the Technique Evaluation dialog's Statistics table. Now press the dialog's Ignore button. All of the selected records will now not be used to calculate statistics and not be displayed in the dialog's graphs. Ignoring the estimates for water will significantly reduce the Avg Abs % Error.
    1
    Pressing the dialog's Ignore button will remove the selected records from statistics calculations and graphs.
    2
    Ignored records are displayed in light gray.
    3
    Ignoring outliers can often significantly improve the technique's accuracy.
    4
    Ignored records are not displayed in the evaluation dialog's graphs. The removal of outlining values often reveals additional outlying data points that should be investigated further.

This particular analysis shows that the Cp,l (T): Missenard Method [MKS] does not estimate the liquid heat capacity of water well. Having obtained this result, we should first examine the technique's implementation to ensure no errors are present, and then, assuming correct implementation, consider add an applicability rule to improve the technique's performance.

Tip: Check manual technique assigments

When evaluating an estimation technique, Cranium and Synapse will use selected manual techniques. We recommend always checking the manual techniques selections and usually clearing all manual technique selections before performing a technique evaluation. (See documentation on the Manual Techniques Dialog for more documentation.)

Example: Adding an applicability rule

Given the results of the evaluation performed in the previous example, we will add an applicability rule that excludes the use of the Cp,l (T): Missenard Method [MKS] technique for estimating the liquid heat capacity of water. (Note: this rule may already be present in the knowledge bases you are using.)

  1. Open a knowledge base document. (Open a "working" document or create a copy of a document (see here) if you are just experimenting with this functionality.)
  2. Change to the Techniques Chapter and navigate to the Cp,l (T): Missenard Method [MKS] technique. (See the Navigation Overview documentation for details on navigating chapters and pages.)
  3. Scroll to the Model Information Section and click the left mouse button on the Accuracy and Applicability Rules field's large edit control. The application will activate the Code Edit dialog. (See here for documentation on this dialog.)
  4. Enter the following code into the dialog's edit control:
    // Default declarations string chemical; // Default assignments chemical = Chemical(); // Exclude certain chemicals if( chemical == "Water" ) return FALSE; // Average absolute % error SetResult(3.1994); // Applicable return TRUE;

    Note that the code's result, i.e., the argument to the SetResult function, is the Avg Abs % Error calculated by the Technique Evaluation dialog.

  5. Finally, press the dialog's Save button to store the new applicability code into the current document.
Tip: Remember that values are case sensitive

Remember that text values in both Cranium and Synapse are case sensitive. Thus, "ethanol" does not equal "Ethanol". If you wish to make your technique's code more robust, you should account for differences in case. For example, the previous applicable code could be written as:

// Default declarations string chemical; // Default assignments chemical = Chemical(); // Exclude certain chemicals if( chemical == "Water" ) return FALSE; if( chemical == "water" ) return FALSE; // Average absolute % error SetResult(3.1994); // Applicable return TRUE;

Related Documentation
Topic Description
Estimating Chemical Properties a short video demonstrating how to estimate the physical properties of pure chemical using either Synapse or Cranium.
Estimating Mixture Properties a short video demonstrating how to estimate the physical properties of mixtures using either Synapse or Cranium.
Getting Started using Cranium provides a quick tour of Cranium's capabilities including physical property estimation and a discussion of structure editing.
Getting Started using Synapse provides a quick tour of Synapse's capabilities including examples of chemical product design.