45-733 PROBABILITY AND STATISTICS I Topic #3

1 February 2000, 3 February 2000

VIII. Expectation and Variance

1. Definition: Expected Value
   E(X) = å_i=1,n x_if(x_i) for discrete
   E(X) = ò_-¥^+¥ xf(x)dx for continuous


2. Example: Discrete Uniform Distribution: Roll one die
   

   
                            æ 1/6  x=1,2,3,4,5,6
                     f(x) = ç
                            è 0 otherwise
   

   
   E(X) = å_i=1,6 x_if(x_i) = 1/6(1 + 2 + 3 + 4 + 5 + 6) = 21/6 = 3.5


3. E(X) is the mean of f(x). In the simple example of one die, E(X) is quite literally the arithmetic average or center of mass or center of balance -- note that 3.5 is the "midpoint" between 1 and 6.


4. Example: Continuous Uniform Distribution
   

   
                            æ 1/(b - a)  a < x < b
                     f(x) = ç
                            è 0 otherwise
   

   
   E(X) = ò_-¥^+¥ xf(x)dx = ò_a^b x[1/(b-a)]dx = [1/(b-a)](x²/2)|_a^b =
   [1/(b-a)][(b² - a²)/2] = (b+a)/2


5. Example: Bernoulli Distribution
   

   
                            æ  p  x = 1
                            ç 
                     f(x) = ç 1-p x = 0
                            ç
                            è  0 otherwise
   

   
   E(X) = å_i=1,2 x_if(x_i) = 1*p + 0*(1 - p) = p


6. Example:
   

   
                            æ 3x2  0 < x < 1
                     f(x) = ç
                            è   0 otherwise
   

   
   E(X) = ò_-¥^+¥ xf(x)dx = ò₀¹ x[3x²]dx = [3x⁴/4]|₀¹ = 3/4


7. Expected Value of a Function of a Random Variable
   Let r(X) be any function of X. For example:
   r(X) = aX + b; where a and b are constants;
   r(X) = X^1/4
   r(X) = e^X; and so on.
   Then
   E(r(X)) = å_i=1,n r(x_i)f(x_i) for discrete
   E(r(X)) = ò_-¥^+¥ r(x)f(x)dx for continuous


8. Example: E(r(X)) = X² for Discrete Uniform Distribution (roll one dice)
   E(X²) = å_i=1,6 (x_i²)*(1/6) = (1/6)(1 + 4 + 9 + 16 + 25 + 36) = 91/6


9. Example: E(r(X)) = X² for Continuous Uniform Distribution
   E(X²) = ò_a^b(x²)(1/[b-a])dx = (1/[b-a])ò_a^b(x²)dx = (1/[b-a])(x³/3)|_a^b =
   (1/3[b-a])(b³ - a³) = (b² + ab + a²)/3


10. Example: E(r(X)) = X² for Bernoulli Distribution
    E(X²) = (1²)*p + (0²)*(1-p) = p


11. Example: E(r(X)) = X² for
    

    
                             æ 3x2  0 < x < 1
                      f(x) = ç
                             è   0 otherwise
    

    
    E(X²) = ò₀¹(x²)3x²dx = ò₀¹3x⁴dx = (3x⁵/5)|₀¹ = 3/5
    


12. Two Important Properties of Expected Values
    
    
    1. E(aX + b) = aE(X) + b
       To see this, let r(X) = aX + b, then:
       E(r(X)) = E(aX + b) = ò_-¥^+¥ (ax + b)f(x)dx =
       ò_-¥^+¥ axf(x)dx + ò_-¥^+¥ bf(x)dx =
       a(ò_-¥^+¥ xf(x)dx) + b(ò_-¥^+¥ f(x)dx) = aE(X) + b
       because the the term, ò_-¥^+¥ xf(x)dx, is just the definition of E(X), and
       ò_-¥^+¥ f(x)dx = 1 by the 2nd Axiom of Probability.
       Note that this shows that the expected value of a constant is a constant!
       That is, E(b) = b, where b is a constant. Because E(X) is a constant, this means that:
       E[E(X)] = E(X)!
       
       
    2. E(X₁ + X₂ + ... + X_n) = E(X₁) + E(X₂) + ... + E(X_n)
       Proof: This will be shown in either Notes #7 or Notes #8.
    
    
13. Variance
    Variance is a measure of the dispersion of a random variable.
    Definition: Variance: VAR(X) = s² = E{[X - E(X)]²}
    Simplifying: VAR(X) = s² = E{X² -2XE(X) + [E(X)]²} =
    E(X²) -2E(X)E(X) + [E(X)]² = E(X²) - [E(X)]²
    Note that we used (12A) to derive this.


14. Standard Deviation
    The standard deviation of a random variable is just the square root of the variance. Namely:
    s = [VAR(X)]^1/2
    The standard deviation is in the same units as the random variable! This is not true of the variance!


15. Example: VAR(X) for Discrete Uniform Distribution (roll one dice)
    From (2) and (8), VAR(X) = s² = 91/6 - (21/6)²


16. Example: VAR(X) for Continuous Uniform Distribution
    From (4) and (9), VAR(X) = s² = (b² + ab + a²)/3 - [(b + a)/2]² = [(b - a)²]/12


17. Example: VAR(X) for Bernoulli Distribution
    From (5) and (10), VAR(X) = s² = p - p² = p(1 - p)


18. Example: VAR(X) for:

    
                             æ 3x2  0 < x < 1
                      f(x) = ç
                             è   0 otherwise
    

    
    From (6) and (11), VAR(X) = 3/5 - (3/4)² = 3/80
    
19. Problem 3.18 p.87
    Let X = "daily sales in dollars". Clearly the range of X is $0, $50,000, $100,000.
    To get P(X = 100,000) note that the salesman would have to contact two customers and sell both customers the heavy-equipment.
    Hence: P(X = 100,000) = (2/3)*(1/10)*(1/10) = 2/300
    To get P(X = 0) the salesman could either contact one customer and make no sale, or contact two customers and make no sales.
    Hence: P(X = 0) = (1/3)*(9/10) + (2/3)*(9/10)*(9/10) = 252/300
    To get P(X = 50,000) the salesman could either contact one customer and make a sale, or contact two customers and make only one sale.
    Hence: P(X = 50,000) = (1/3)*(1/10)+ (2/3)*[(1/10)*(9/10) + (9/10)*(1/10)] = 46/300
    This allows us to state the distribution of X:
    

    
    
                              æ 252/300  x = 0  
                              ç
                              ç  46/300  x = $50,000  
                       f(x) = ç
                              ç   2/300  x = $100,000  
                              ç
                              è  0 otherwise
    

    
    
    1. To get the mean of daily sales:
       E(X) = å_i=1,3 x_if(x_i) = 0*(252/300) + 50,000*(46/300) + 100,000*(2/300) = $8333.00
    
    
    2. To get the standard deviation, s, of daily sales, we calculate the variance and take its square root.
       E(X²) = å_i=1,3 (x_i)²f(x_i) = (0²)*(252/300) + (50,000²)*(46/300) + (100,000²)*(2/300) = 450,000,000
       VAR(X) = s² = 450,000,000 - (8333²) = 380,561,111
       So that: std. dev. = [VAR(X)^1/2] = s = $19,507.98
       Recall that the standard deviation is in the original units of X and can be so interpreted! This is not true of VAR(X).
    
    
20. Two Important Properties of Variances
    
    
    1. VAR(aX + b) = (a²)VAR(X) = (a²)s²
       To see this, let r(X) = aX + b, then:
       VAR[r(X)] = E{[aX + b - E(aX + b)]²} = E{[aX + b - aE(X) - b]²} =
       E{[a²][X - E(X)]²} = (a²)E{[X - E(X)]²} = (a²)VAR(X) = (a²)s²
       Note that (12a) of notes #5 was used to derive this.
       
    2. If X₁, X₂, ... , X_n, are independent random variables, then:
       VAR(a₁X₁ + a₂X₂ + ... + a_nX_n + b) = å_i=1,n (a_i)²VAR(X_i) = å_i=1,n (a_i)² s_i²
       See discussion in Notes #8.
    
    
21. Problem 4.19 p.149
    By inspection, we see that this is a continuous uniform distribution with parameters:
    b = 61 and a = 59. Hence:
    E(X) = (b + a)/2 = 60 and VAR(X) = s² = [(b - a)²]/12 = 4/12 = 1/3


22. Problem 4.20 p.149
    

    
                             æ 2y  0 < y < 1
                      f(y) = ç
                             è 0 otherwise
    

    
    
    1. Find the mean and the variance:
       E(Y) = ò₀¹ y*2ydy = (2(y³)/3)|₀¹ = 2/3
       E(Y²) = ò₀¹ (y²)*2ydy = (2y⁴/4)|₀¹ = 1/2
       Hence: VAR(Y) = s² = (1/2) - (2/3)² = 1/18
       
    2. Let the profit be equal to: X = 200Y - 60. Find the mean and the variance of X.
       E(X) = E(200Y - 60) = 200E(Y) - 60 = 200*(2/3) - 60 = 220/3
       VAR(X) = VAR(200Y - 60) = (200²)VAR(Y) = 20,000/9
       
    3. To solve c) we need the Tchebysheff Inequality. Since I am not teaching it, we will skip this!