Discovering Patterns
PDT's early work centered on the principles of Information Theory. PDT Founder and Chairman Dr. Andrew Wong pioneered the application of Information Theory to statistical inference.
The basic principle of Information Theory is that the amount of information in a set of data can be measured. The algorithms developed by PDT rigorously measure this information – specifically, we compare the frequency of occurrence of all possible combinations of factors to the frequency that we would expect to occur by chance. The difference in information between the measured frequency and the expected frequency is called the residual. Relatively high residuals indicate that a particular combination of factors are happening far more often or far less often in real operations than would normally be expected.
To speed up these calculations in large, disparate data sets, the algorithm smartly truncates impossible combinations from the investigation. The result is a clean set of statistically significant events indicating strong and measurable correlations between factors.
Statistical significance is measured by placing the residuals along a normal distribution curve, transforming and ranking them into adjusted residuals. At this point in the analysis, no cause and effect claims can be made but there is statistical proof from the data itself that strong inter-relationships exist. Furthermore, the algorithm delivers descriptive patterns in plain english that domain experts can incorporate into their efforts to gain a better understanding of these hidden, high order inter-relationships and their impact on any chosen output.
