Computational Mechanics provides a precise theory of how natural processes can be represented in an “optimal way”, i.e. as an epsilon machine. “Optimal” here means as “compact” as possible and as “exact” as possible. “Compactness” is measured as and “exactness” is measured as . Naturally, you might ask: Why are these measures of an “optimal” representation?
Inference then encompasses the actually formidable task of actually finding such an optimal representation, which is all but trivial.
How much data is needed to determine the causal states? The optimality theorems for causal states hold for infinite histories and infinite futures - do we really need this? What can we prove about finite histories and finite futures?