Monday, September 20, 2010

Introduction to probability, ch#3(General Random Variables) notes - Part-1

Introduction to probability, ch#3(General Random Variables) notes - Part-1

A "continuous" random variable can take any value on the real number line unlike a discrete random variable.

PDF(Probability Density Function), counterpart of PMF of discrete random variables, of a continuous random variable X is a nonnegative function fX s.t.
P(XB) = BfX(x)dx
for every subset of the real line. In particular, the probability that the value of X falls within an interval is
P(aXb) = abfX(x)dx.

Note that, for a single value a, P(X = a) = aafX(x)dx = 0. Hence, excluding or including the endpoints of any interval has no effect on its probability. That is
P(aXb) = P(a<X<b) = P(aX<b) = P(a<Xb)

PDF also has to satisfy the normalization property, -fX(x)dx = 1

Also, note that PDF is not the probability of any particular event and hence not necessarily restricted to less than or equal to 1. It just has to be nonnegative and satisfy normalization property stated above.

-- Expectation and Variance: --
Let X be a continuous random variable with PDF fX(x)

E[X] = -xfX(x)dx
E[g(X)] = -g(x)fX(x)dx

var(X) = E[(X - E[X])^2] = -(x-E[X])2fX(x)dx = E[X2]-(E[X])2

If, Y = aX + b then
E[Y] = aE[X] + b
var(Y) = a2var(X)


-- Cumulative Distribution Function --
CDF is a unified concept to deal with discrete as well as continuous random variables. CDF of X is denoted by FX.
FX(x) = P(Xx) = kxpX(k) , if X is discrete
FX(x) = P(Xx) = -xfX(t)dt, if X is continuous

Conceptually, FX(x) "accumulates" probability "upto" the value x.

If X is discrete, then we can obtain PMF of X from CDF of X as follow
pX(k) = P(Xk) - P(Xk-1) = FX(k) - FX(k-1), for all integers k

If X is continuous, then the PDF can be obtained by differentiating the CDF(assuming it has a derivative)
fX(x) = dFXdx(x)

-- Conditioning on an event --
Let A be some event, then conditional PDF of X, fXA is defined as follow
P(XBA) = BfXA(x)dx

in the special case when conditioning event is of the form XA, we have
P(XBXA) = P(XB,XA)P(XA) = ABfX(x)dxP(XA)

so, BfXAdx = ABfX(x)dxP(XA)

Hence, for events of the type XA
fXA = fX(x)P(XA), if xA and 0 otherwise

A version of total probability theorem:
Let A1,A2,...,An form partition of the sample space with P(Ai) for each i, then
fX(x) = i=1nP(Ai)fXAi(x)

Conditional Expectation
E[X|A] = -xfXA(x)dx
E[g(X)|A] = -g(x)fXA(x)dx

total expectation theorem:
E[X] = i=1nP(Ai)E[XAi]
E[g(X)] = i=1nP(Ai)E[g(X)Ai]

No comments:

Post a Comment