bijnum
Class BIJstats

java.lang.Object
  |
  +--bijnum.BIJstats

public class BIJstats
extends java.lang.Object

BIJ statistical methods inherited from MatLab. Syntax is as close as possible to matlab. Copyright (c) 1999-2003, Michael Abramoff. All rights reserved.


Field Summary
static float CI95
          Conversion from standard error to confidence intervals at different significance levels.
static float CI99
           
static float CI99_9
           
 
Constructor Summary
BIJstats()
           
 
Method Summary
static double accuracy(float[] table2x2)
          Compute accuracy or observed proportion of overall agreement of a test for which the result is in exp and the ground truth in truth accuracy = true pos + true neg / (true neg + false pos + true pos + false neg) accuracy = (a+d) / (a+b+c+d)
static double accuracyMultipleClasses(float[] exp, float[] truth, int n)
          Compute the accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.
static double accuracyMultipleClasses(int[][] tablenxn)
          Compute the accuracy of a test of which the confusion matrix is in tablenxn.
static double[] autocorr(double[] data, int maxk)
          Computes autocorrelation up to maxk for the time series data.
static double[] autocov(double[] data, int maxk)
          Computes autocovariance up to maxk for the time series data.
static double avg(double[] v)
          Compute average of all values in vector v.
static double avg(float[] v)
          Compute average of all values in vector v.
static double avg(float[][] m)
          Compute average of all values in vector v.
static float avg(float[] v, float[] mask)
           
static float avgNoExtremes(float[] v, float nrstddev)
           
static int binIndex(int[] histogram, float fraction)
          Compute the lowest bin into which the highest p percent of occurrences falls.
static float correl(float[] a, float[] b)
          Compute the correlation of a vector with another vector b.
static float[][] covariance(float[][] m, boolean doShowProgress)
          Compute the covariance matrix for a matrix of size NxM.
static double erf(double x)
          Compute the erf of x.
static double getSignificanceLevel(double significance)
           
static java.lang.String getSignificanceString(double sign)
           
static int[] histogram(float[] v, float min, float d, int n)
          Compute a histogram with n bins for the vector v, each bin separated by d.
static double kappa(float[] table2x2)
          Compute kappa for a 2x2 table.
protected static double[] linearcorrect(double[] data)
          Corrects data for a linear trend.
static float[] meanColumn(float[][] m)
           
static float[] meanColumnT(float[][] m)
           
static double means(float[][] m)
           
static float mse(float[] a, float[] b)
          Compute Mean Square Error (or residual) of vectors a and b.
static int n(float[] v)
           
static int n(float[] v, float[] mask)
           
static float[] randomFraction(float[] v, double fraction)
          Randomly sample a fraction of the elements of vector v.
static float[] residuals(float[] a, float[] b)
          Compute residuals of b versus a.
static float rmse(float[] a, float[] b)
          Compute Root Mean Square Error of vectors a and b.
static double sem(double estimateAB, double estimateA, double semA, double estimateB, double semB)
          Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1.
static double sem(float[] v)
          sem(1) computes the standard error of the mean of the elements of v.
static double sem(float[] v, float[] mask)
          sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0.
static float[] sensitivities(float[] truth, float[] exp, int n)
          Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
static double sensitivity(float[] table2x2)
          Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c.
static float[] specificities(float[] exp, float[] truth, int n)
          Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
static double specificity(float[] table2x2)
          Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c.
static float[] spectrum(float[] v)
          Compute the spectrum of v.
static float stdev(float[] v)
          Compute stdev (SQRT(var)) of all values in vector v.
static float stdev(float[] v, float[] mask)
          stdev(2) computes the stddev of the elements of only where the mask value is not 0.
static double sum(double[] v)
          Compute average of all values in vector v.
static double sum(float[] v)
          sum(1) computes summation of all values in vector v.
static double sum(float[][] m)
          sum(1) computes summation of all values in matrix m.
static double sum(float[] v, float[] mask)
          sum(2) computes the sum of the elements of vector v only where the mask value is not 0.
static float[] table2x2(float[] obs0, float[] obs1, int c)
          Make a 2x2 table for the observations obs0 and obs1, where an observation c is taken as correct.
static int[][] tablenxn(float[] obs0, float[] obs1, int n)
          Make a nxn confusion matrix for the observations obs0 and obs1, for n classes.
static float thresholdFraction(float[] v, double fraction)
           
static float thresholdFraction(float[] v, float fraction)
          Find the value in vector over which fraction of all values in vector lie.
static double tptest(double[] data1, double[] data2)
          Perform paired Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
static double ttest(double[] data1, double[] data2)
          Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
static double ttest(float[] data1, float[] data2)
          Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
static float[] unitvar(float[] v)
          Make a new vector of v that has zero mean and unit variance.
static void unitvar(float[][] m)
          Make m unit variance and zero mean with respect to all elements of m.
static double var(double[] v)
          Compute variance of all elements in vector v.
static double var(float[] v)
          Compute variance of all elements in vector v.
static double[] zeromean(double[] v)
          Zero the mean of all elements of v.
static float[] zeromean(float[] v)
          Zero the mean of all elements of v.
 
Methods inherited from class java.lang.Object
, clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CI95

public static float CI95
Conversion from standard error to confidence intervals at different significance levels. To make a confidence interval (99%, 2.576; 95%, 1.96; 99.9 3.39).

CI99

public static float CI99

CI99_9

public static float CI99_9
Constructor Detail

BIJstats

public BIJstats()
Method Detail

getSignificanceLevel

public static double getSignificanceLevel(double significance)

covariance

public static float[][] covariance(float[][] m,
                                   boolean doShowProgress)
Compute the covariance matrix for a matrix of size NxM. (= m (TRANSPOSE(m)/length(m)) The returned matrix will be size NxN.
Parameters:
m - a matrix of float[N][M]
Returns:
a matrix of covariances float[N][N]

n

public static int n(float[] v)

n

public static int n(float[] v,
                    float[] mask)

avg

public static float avg(float[] v,
                        float[] mask)

getSignificanceString

public static java.lang.String getSignificanceString(double sign)

stdev

public static float stdev(float[] v,
                          float[] mask)
stdev(2) computes the stddev of the elements of only where the mask value is not 0. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.
Parameters:
v - a float[] vector.
mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:
the stddev of the masked elements in v.

sem

public static double sem(float[] v)
sem(1) computes the standard error of the mean of the elements of v. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.
Parameters:
v - a float[] vector.
Returns:
the standard error of the mean of all elements in v.

sem

public static double sem(float[] v,
                         float[] mask)
sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.
Parameters:
v - a float[] vector.
mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:
the stderr of the masked elements in v.

sem

public static double sem(double estimateAB,
                         double estimateA,
                         double semA,
                         double estimateB,
                         double semB)
Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1. The combined SEM is used to compute the SEM correspnding to the multiplication or division of two estimates. The method used to calculate the SE of a calculated result involving multiplication and/or division is similar to, but slightly more complicated than that used for addition and subtraction. In this case, the relative standard deviations (or relative standard errors) of the values involved in the calculation must be used rather than the absolute standard errors. The relative SD (or SE) is obtained from the absolute SD (or SE) by dividing the (SE or SD) by the data value itself. Note that these relative quantities are always unitless.
Parameters:
estimateAB - the estimate of the combination A and B (determined by either multiplication or division, don't care here).
estimateA, - semA the estimate and standard deviation of estimate.
estimateB, - semBV the estimate and standard deviation of estimate.
Returns:
standard error of the mean for estimateAB.

avgNoExtremes

public static float avgNoExtremes(float[] v,
                                  float nrstddev)

meanColumnT

public static float[] meanColumnT(float[][] m)

meanColumn

public static float[] meanColumn(float[][] m)

means

public static double means(float[][] m)

avg

public static double avg(float[] v)
Compute average of all values in vector v.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

avg

public static double avg(float[][] m)
Compute average of all values in vector v.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

avg

public static double avg(double[] v)
Compute average of all values in vector v.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

sum

public static double sum(float[] v)
sum(1) computes summation of all values in vector v.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

sum

public static double sum(float[][] m)
sum(1) computes summation of all values in matrix m.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

sum

public static double sum(float[] v,
                         float[] mask)
sum(2) computes the sum of the elements of vector v only where the mask value is not 0.
Parameters:
v - a float[] vector.
mask - a float[] with a value of ! 0 for all elements of v that are valid.
Returns:
the avg of the masked elements in v.

sum

public static double sum(double[] v)
Compute average of all values in vector v.
Parameters:
v - a vector of float[]
Returns:
the average of all elements in v.

var

public static double var(float[] v)
Compute variance of all elements in vector v.
Parameters:
v - a vector of float[]

unitvar

public static void unitvar(float[][] m)
Make m unit variance and zero mean with respect to all elements of m. m is modified in place.
Parameters:
m - a vector of float[][]

var

public static double var(double[] v)
Compute variance of all elements in vector v.
Parameters:
v - a vector of float[]

unitvar

public static float[] unitvar(float[] v)
Make a new vector of v that has zero mean and unit variance.
Parameters:
v - a float[] vector
Returns:
a vector with proportionally the same elements as v but with zero mean and unit variance.

zeromean

public static double[] zeromean(double[] v)
Zero the mean of all elements of v.
Parameters:
v - a vector.
Returns:
a vector with zero mean and same variance as v.

zeromean

public static float[] zeromean(float[] v)
Zero the mean of all elements of v.
Parameters:
v - a vector.
Returns:
a vector with zero mean and same variance as v.

stdev

public static float stdev(float[] v)
Compute stdev (SQRT(var)) of all values in vector v.
Parameters:
v - a vector of float[]

thresholdFraction

public static float thresholdFraction(float[] v,
                                      double fraction)

thresholdFraction

public static float thresholdFraction(float[] v,
                                      float fraction)
Find the value in vector over which fraction of all values in vector lie.
Parameters:
vector - a vector of values
fraction - a float [0-1].

histogram

public static int[] histogram(float[] v,
                              float min,
                              float d,
                              int n)
Compute a histogram with n bins for the vector v, each bin separated by d.
Parameters:
v - a float[] vector
min - the lowest value for the first bin.
d - the difference in value between each bin
n - the number of bins.
Returns:
a float[] with the number of occurences for each bin value.

binIndex

public static int binIndex(int[] histogram,
                           float fraction)
Compute the lowest bin into which the highest p percent of occurrences falls. Used to compute things as "give me the occurence above which 10% of the histogram falls."
Parameters:
histogram - an int[] with the ouccrence counts for each bin
p - the fraction of histogram values desired.
Returns:
an int, the index into the lowest bin that still falls among p.

correl

public static float correl(float[] a,
                           float[] b)
Compute the correlation of a vector with another vector b. The mean of a and b is subtracted before further processing. a and b therefore do not have to be zero-mean.
Parameters:
a - a float[] vector.
b - a float[] vector of same length.
Returns:
r, the correlation coefficient.

spectrum

public static float[] spectrum(float[] v)
Compute the spectrum of v. The spectrum gives the ratio of each element to v to the sum of v. The variance spectrum gives the ratio of each coordinate to the sum of all coordinates.
Parameters:
v - a float[] vector
Returns:
the spectrum of v as a float[].

erf

public static double erf(double x)
Compute the erf of x.
Parameters:
x - the argument
Returns:
the erf of x.

mse

public static float mse(float[] a,
                        float[] b)
                 throws java.lang.IllegalArgumentException
Compute Mean Square Error (or residual) of vectors a and b. a and b are normalized to zero mean and unit variance.
Parameters:
a - a float[] vector
b - a float[] vector.
Returns:
a float that is the Mean Square Error of a and b.

residuals

public static float[] residuals(float[] a,
                                float[] b)
                         throws java.lang.IllegalArgumentException
Compute residuals of b versus a.
Parameters:
a - a float[] vector
b - a float[] vector.
Returns:
a float[] that contains the residuals for each b-a.

rmse

public static float rmse(float[] a,
                         float[] b)
                  throws java.lang.IllegalArgumentException
Compute Root Mean Square Error of vectors a and b. a and b are normalized to zero mean and unit variance.
Parameters:
a - a float[] vector
b - a float[] vector.
Returns:
a float that is the Root Mean Square Error of a and b.

ttest

public static double ttest(float[] data1,
                           float[] data2)
Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same. From Press, Numerical Recipes in C, second edition.
Parameters:
data1 - array of datapoints
data2 - array of datapoints
t -  
prob -  

autocov

public static double[] autocov(double[] data,
                               int maxk)
Computes autocovariance up to maxk for the time series data.

linearcorrect

protected static double[] linearcorrect(double[] data)
Corrects data for a linear trend.

autocorr

public static double[] autocorr(double[] data,
                                int maxk)
Computes autocorrelation up to maxk for the time series data.

ttest

public static double ttest(double[] data1,
                           double[] data2)
Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same. From Press, Numerical Recipes in C, second edition.
Parameters:
data1 - array of datapoints
data2 - array of datapoints

tptest

public static double tptest(double[] data1,
                            double[] data2)
Perform paired Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same. From Press, Numerical Recipes in C, second edition, pp 618
Parameters:
data1 - array of datapoints
data2 - array of datapoints

randomFraction

public static float[] randomFraction(float[] v,
                                     double fraction)
Randomly sample a fraction of the elements of vector v.
Parameters:
v - a vector
fraction - the fraction of elements from v to be included.
Returns:
a float[] vector with all the sampled elements.

sensitivities

public static float[] sensitivities(float[] truth,
                                    float[] exp,
                                    int n)
                             throws java.lang.IllegalArgumentException
Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
Parameters:
exp - a float[] vector of test results, where 0 <= exp[n] < n.
truth - a float[] vector of ground truth, where 0 <= truth[n] < n.
n - the number of classes to determine the sensitivity for.
Returns:
a float[] vector with the sensitivities for all classes in truth.

specificities

public static float[] specificities(float[] exp,
                                    float[] truth,
                                    int n)
                             throws java.lang.IllegalArgumentException
Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
Parameters:
exp - a float[] vector of test results, where 0 <= exp[n] < n.
truth - a float[] vector of ground truth, where 0 <= truth[n] < n.
n - the number of classes to determine the sensitivity for.
Returns:
a float[] vector with the specificities for all classes in truth.

table2x2

public static float[] table2x2(float[] obs0,
                               float[] obs1,
                               int c)
                        throws java.lang.IllegalArgumentException
Make a 2x2 table for the observations obs0 and obs1, where an observation c is taken as correct. The table is Obs0==c Obs0!=c Obs1==c a b Obs1!=c c d The table will contain a,b,c,d in that order
Returns:
the 2x2 table.

tablenxn

public static int[][] tablenxn(float[] obs0,
                               float[] obs1,
                               int n)
                        throws java.lang.IllegalArgumentException
Make a nxn confusion matrix for the observations obs0 and obs1, for n classes. The table is Obs0==0 Obs0==1 Obs0==2 ... Obs0==n-1 Obs1==0 a b c ... Obs1==1 g h ... Obs1==2 l m Obs1==n-1 ..
Returns:
the nxn confusion matrix.

kappa

public static double kappa(float[] table2x2)
Compute kappa for a 2x2 table. Kappa is
Returns:
the kappa.

accuracyMultipleClasses

public static double accuracyMultipleClasses(float[] exp,
                                             float[] truth,
                                             int n)
                                      throws java.lang.IllegalArgumentException
Compute the accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.
Parameters:
exp - an int[] vector of test results, where 0 <= exp[n] < n.
truth - an int[] vector of ground truth, where 0 <= truth[n] < n.
n - the number of classes to determine the sensitivity for.
Returns:
the combined accuracy.

sensitivity

public static double sensitivity(float[] table2x2)
                          throws java.lang.IllegalArgumentException
Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c. sensitivity = true pos / (true pos + false neg) sensitivity = a / (a+c)
Parameters:
exp - a float[] vector of test results, where 0 <= exp[n] <= c.
truth - a float[] vector of ground truth, where 0 <= truth[n] <= c.
c - the class to determine sensitivity for.
Returns:
the sensitivity.

specificity

public static double specificity(float[] table2x2)
                          throws java.lang.IllegalArgumentException
Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c. specificity = true neg / (true neg + false pos) specificity = d / (b+d)
Parameters:
exp - an int[] vector of test results, where 0 <= exp[n] <= c.
truth - an int[] vector of ground truth, where 0 <= truth[n] <= c.
c - the class to determine specificity for.
Returns:
the specificity.

accuracy

public static double accuracy(float[] table2x2)
                       throws java.lang.IllegalArgumentException
Compute accuracy or observed proportion of overall agreement of a test for which the result is in exp and the ground truth in truth accuracy = true pos + true neg / (true neg + false pos + true pos + false neg) accuracy = (a+d) / (a+b+c+d)
Parameters:
table2x2 - a 2x2 table.
Returns:
the accuracy.

accuracyMultipleClasses

public static double accuracyMultipleClasses(int[][] tablenxn)
                                      throws java.lang.IllegalArgumentException
Compute the accuracy of a test of which the confusion matrix is in tablenxn.
Returns:
the combined accuracy.