Consider a random sampling process in which all the outcomes solely depend on the chance, i.e., each outcome is equally likely to happen. If the collection of all possible outcomes is U and the collection of desired outcomes is A, the probability of the desired outcomes is:

Since A is a subset of U (see Set Theory), 0<=n(A)<=n(U), the probability of the desired outcomes is:


Accordingly, the probability of an unwanted outcome !A is:

Important Properties

Join Probability: The probability of the desired outcomes is the sum of the probability of each event resulting in a desired outcome.

Suppose that the set of desired outcomes A has M different events, i.e., A={am|m=1,2,3,...,M}. The join probability indicates that :

Conditional Probability: Suppose that A and B are two sets of outcomes. The probability of B under the condition that A has happened, denoted by P(B|A), can be expressed as:

Independency: Suppose that A and B are two sets of outcomes. If the probability of B was not affected by whether A has happened or not, A and B are two independent sets of outcomes. Combining the conditional probability and independency, we have

General Terms

Density Function: A probability density function f(x) gives the probability of each possible outcome x.

Distribution Function: A probability distribution function F(x) gives the probability of all possible outcomes accumulated from the reference outcome (starting point) up to the current outcome x. The probability density function and the probability distribution function have the following relationship:

Discrete System Continuous System

Mean Value: Mean Value, mean, or expectation, denoted by mu, is the likely outcome in an average sense. The average of all outcomes in a large-sample random sampling process is expected to be (close to) the mean value which is defined as:

Discrete System Continuous System

Variance: Variance denoted by sigma^2 gives the spread of a distribution measured from the likely outcome mu. It is defined as:

Discrete System Continuous System

Standard Deviation: Standard deviation, denoted by , is the positive square root of the variance. Both variance and standard deviation are used to describe the spread of a distribution.

Discrete System Continuous System

• For further details on the Probability Density Functions, Probability Distribution Functions, Mean Values, and Variances, please see the Distributions section.