Ill Posed Problem
A problem is ill-posed if it fails to meet one or more of the three conditions of a well-posed problem defined by Hadamard:
- Existence: A solution exists.
- Uniqueness: The solution is unique.
- Stability (continuous dependence): the solution changes continuously with small changes in the input.
Intuition: theoretically solutions exists but tiny disturbances makes them unstable.
Examples:
- ML1 - Well and Ill Posed Problems → “classification is an example of ill posed problem”
- Computer Vision is another ill posed problem
- Inverse problems like deblurring an image because small noise can completely change the reconstructed signal.
- Inverting a covariance matrix of features when the number of features exceeds the number of samples
is ill-posed. Since the matrix may be singular or nearly singular making solutions unstable.