Probability spaces

A probability space represents the possible results of a random experiment or process, called events, each one of which may happen with a certain probability. For example, the result of throwing two dice may be the event “seven turns out” or “a pair number turns out”.

Hence, it can be seen that the events of a probability space may be elementary or composed out of elementary events. An elementary event, in the former example, is “(2,5) turns out”; this event is a constituent of the complex event “seven turns out”. In general, each complex event is identified with the set of all the elemen­tary events that constitute it, the occurrence of any of these being sufficient for the occurrence of the complex event. Elementary events are called sample points and the set of all the sample points is called sample space.

From a logical point of view, ‘sample space’ is a primitive term which is char­acterized axiomatically merely as a nonempty set S, and ‘sample point’ is defined just as any element of the sample space. The space of events, which is the family of complex events, is also introduced as a primitive term and characterized as an algebra of sets over S. The third primitive is ‘probability measure’, axiomatically characterized as a measure over the algebra of events. The canonical definition of ‘probability space’ is the following.

12.4.1 Definition

Recall that a σ-algebra of sets over S is a ring of sets, with S as unit, closed under countable unions. A compact way of formulating axiom 3 consists of saying that P is a measure over F.

The following are elementary identities that hold in every probability space. Their proof is easy and is left to the reader.

12.4.2 Theorem

The sample space of a probability space may be finite, countably infinite, or con­tinuous. If S is at most countable, at least some elementary events must get positive probabilities. This is done through a function called ‘probability distribution’.

Table 12.1 Correlations between set-theoretic and probabilistic notions

12.4.3 Definition

A probability distribution assigns positive probabilities at least to some elemen­tary events. If it assigns positive probabilities to all of them it is called a simple distribution.

It is easy to see that a probability distribution over a set S generates in a natural way a probability space if, for each subset A of S, P(A) is de fined by means of equality

As space of events it is usual to take the power set of S.

A σ-algebra particularly important in probability theory is the minimum set containing all the intervals of R which is closed under countable unions. This algebra is called Borel algebra, denoted by B, and its elements are called Borel sets. These notions are essential for the definition of the concept of a random variable.

12.4.4 Definition

6 = (S, F, P) is a countably additive probability space iff 6 is a probability space such that

A probability space whose sample space S is at most countable is called dis­crete. It is called continuous if the probability of any elementary event is zero.

12.5