Thursday, September 16, 2010

Introduction to Probability - Ch#1 notes

Introduction to Probability, Ch#1 notes

A probabilistic model is a mathematical description of an uncertain situation. Every probabilistic model involves an underlying process, called the experiment, that will produce exactly one out of several possible (mutually exclusive)outcomes. The set of *all* possible outcomes is called the sample space of the experiment, and is denoted by Ω. A subset of the sample space, a collection of possible outcomes, is called an event.
The probability law assigns to any event A, a nonnegative P(A)(called the probability of A) that encodes our knowledge or belief about the collective "likelihood" of the elements of A. The probability law *must* satisfy following probability axioms

Nonnegativity: P(A) 0, for every event A.
Additivity: If A and B are two disjoint events, then the probability of their union satisfies, P(AB) = P(A) + P(B) . It can be generalized to any number of disjoint events.
Normalization: P(Ω) = 1

-- Conditional Probability --
The conditional probability of an even A, given an event B with P(B) > 0, is defined by P(A|B) = P(AB)P(B) and specifies a new (conditional) probability law on the same sample space Ω. And, All the probability axioms remain valid on the conditional probability law.

Multiplication Rule:
Assuming that all of the conditioning events have positive probability, we have
P(i=1nAi) = P(A1)P(A2A1)P(A3A1A2)...P(Ani=1n-1Ai)

Total Probability Theorem:
Let A1,A2,...,An be disjoint events that form a partition of the sample space and assume that probability of all these events is greater than 0. Then, for any event B, we have
P(B) = P(A1)P(BA1) + P(A2)P(BA2) + ... + P(An)P(BAn)

Bayes' Rule:
Let A1,A2,...,An be disjoint events that form a partition of the sample space and assume that probability of all these events is greater than 0. Then, for any even B such that P(B) > 0, we have
P(AiB) = P(Ai)P(BAi)i=1nP(Ai)P(BAi)

-- Independence: --
Two evens A and B are independent if and only if, P(AB) = P(A)P(B). This can be generalized to any number of elements.
Also, as a consequence P(A|B) = P(A), provided P(B) > 0

Conditional Independence:
Two events A and B are said to be conditionally independent given another event C with P(C) > 0, if
P(ABC) = P(A|C)P(B|C).
And, if in addition P(BC) > 0, then conditional independence is equivalent to the condition
P(A |BC) = P(A|C)

Note that, independence does not imply conditional independence and viceversa

No comments:

Post a Comment