Bayesian Learning

Bayesian Learning

Exercise 4.1 (Reservation Wage)

Prove that

\[\begin{align*} \int_{w_r}^{\infty}(w-w_r)dH(w)=\int_{w_r}^{\infty}[1-H(w)]dw. \end{align*}\]

Solution

We start by simplifying the left-hand side. Let \(F(w) = w - w_r\), the integration by parts formula implies that

\[\begin{align*} \int_{w_r}^{\infty} (w - w_r) dH(w) &= \int_{w_r}^{\infty} F(w) dH(w)\\ &=\left. F(w) H(w) \right|_{w_r}^{\infty} - \int_{w_r}^{\infty} H(w) dF(w)\\ &=\left. (w - w_r)H(w) \right|_{w_r}^{\infty} - \int_{w_r}^{\infty} H(w) dw. \end{align*}\]

Since \(\left. (w - w_r)H(w) \right|_{w_r}^{\infty}=\int_{w_r}^{\infty} dw\), the expression above is indeed equal to \(\int_{w_r}^{\infty} [1-H(w)] dw\).

Exercise 4.2 (Normal-Normal conjugacy)

Let \(y\) be a random variable that is drawn from a normal distribution with mean \(\mu\) and standard deviation \(\sigma\).

1. Consider a dataset \(Y=(y_1,y_2,...,y_n)\) containing \(n\) realizations of \(y\). Write down the likelihood of \(Y\) as a function of \(\mu\) and \(\sigma\)

2. Assume that \(\mu\) itself is Normally distributed around some mean \(\theta\) with standard deviation \(\tau\). Show that upon observing \(Y\), the posterior belief about \(\mu\) remains normally distributed. Derive the posterior mean and standard deviation of \(\mu\). What do you notice about the standard deviation? Comment your finding.

Solution

1. The likelihood of \(Y\) can be written as

\[\begin{align*} L\left(\mu, \sigma | Y \right) &= \prod_{i = 1}^{n} f\left(y_{i}|\mu, \sigma\right) \\ &= \prod_{i = 1}^{n} \frac{1}{\sqrt{2\pi \sigma^{2}}}\text{exp}\left(-\frac{\left(y_{i}-\mu\right)^{2}}{2\sigma^{2}}\right). \end{align*}\]

2. By Bayes’ rule, we find the posterior distribution of \(\mu\) given the data \(Y\):

\[\begin{align*} p\left(\mu|Y,\theta,\tau,\sigma\right) &\propto p\left(Y|\mu,\sigma\right)p\left(\mu|\theta,\tau\right)\\ & \propto \text{exp}\left(-\frac{1}{2}\left[\frac{1}{\sigma^{2}}\sum_{i=1}^{n}\left(y_{i}-\mu\right)^{2} + \frac{\left(\mu-\theta\right)^{2}}{\tau^{2}}\right]\right). \end{align*}\]

Let \(\overline{y} = \frac{1}{n}\sum_{i=1}^{n}y_{i}\), by the Law of Total Variance, we have

\[\begin{equation*} \sum_{i=1}^{n}\left(y_{i}-\mu\right)^{2} = \sum_{i=1}^{n}\left(y_{i} - \overline{y}\right)^{2} + n\left(\overline{y}-\mu\right)^{2}. \end{equation*}\]

Thus the posterior simplifies to:

\[\begin{equation*} p\left(\mu|Y,\theta,\tau,\sigma\right) \propto \text{exp}\left(-\frac{1}{2}\left[\frac{n\left(\overline{y}-\mu\right)^{2}}{\sigma^{2}} + \frac{\left(\mu-\theta\right)^{2}}{\tau^2}\right]\right). \end{equation*}\]

The exponent can be written as:

\[\begin{equation*} \left(\mu - \frac{\frac{\overline{y}}{\sigma^{2}} + \frac{\theta}{\tau^{2}}}{\frac{n}{\sigma^{2}} + \frac{1}{\tau^{2}}}\right)^{2}*\left(\frac{n}{\sigma^{2}} + \frac{1}{\tau^{2}}\right). \end{equation*}\]

From this, we can conclude that the posterior mean \(\mu_{post}\) and the posterior variance \(\sigma^{2}_{post}\) read

\[\begin{align*} & \mu_{post} = \frac{\frac{\overline{y}}{\sigma^{2}} + \frac{\theta}{\tau^{2}}}{\frac{n}{\sigma^{2}} + \frac{1}{\tau^{2}}}, \\ & \sigma^{2}_{post} = \frac{1}{\frac{n}{\sigma^{2}} + \frac{1}{\tau^{2}}}. \end{align*}\]

Note that the posterior variance is smaller than the prior variance, reflecting the fact that, after observing the data, we have more precise information about \(\mu\).

Exercise 4.3 (Bernouilli)

Let \(y\) be a random variable that is drawn from a Bernouilli distribution with probability of success \(p\).

1. Consider a dataset \(Y=(y_1,y_2,...,y_n)\) containing \(n\) realizations of \(y\). Write down the likelihood of \(Y\) as a function of \(p\).

2. Assume that \(p\) itself is drawn from the following distribution

\[\begin{equation*} p=\left\{ \begin{array}{l} p_{h}\text{ with probability } \mu _{0}, \\ p_{l}\text{ with probability } 1-\mu _{0}. \end{array}% \right. \end{equation*}\]

The variable \(\mu_0\) denotes the prior in period \(0\). Use Bayes rule to express the agent’s posterior \(μ_1(0)\) at the beginning of period \(2\) after having observed a failure in period \(1\), i.e. \(y_1=0\). Iterate the computation to derive the posterior after \(n\) failures in a row.

Solution

1. The likelihood of \(Y\) reads

\[\begin{equation*} L\left(p | Y \right) = \prod_{i = 1}^{n} p^{y_{i}}(1-p)^{1-y_{i}}. \end{equation*}\]

2. By Baye’s rule, we can express \(μ_1(0)\) as:

\[\begin{align*} μ_1(0) = p(p_{h}|y_{1}=0) & = \frac{p(y_{1}=0|p_{h})\mu_{0}}{p(y_{1}=0|p_{h})\mu_{0} + p(y_{1}=0|p_{l})(1-\mu_{0})}\\ & = \frac{(1-p_{h})\mu_{0}}{(1-p_{h})\mu_{0} + (1-p_{l})(1-\mu_{0})}. \end{align*}\]

The posterior after \(n\) failures could be expressed as:

\[\begin{align*} μ_n(0^n) = p(p_{h}|y_{1}=0, y_{2}=0,...,y_{n}=0) & = \frac{p(y_{1}=0, y_{2}=0,...,y_{n}=0|p_{h})\mu_{0}}{p(y_{1}=0, y_{2}=0,...,y_{n}=0|p_{h})\mu_{0} + p(y_{1}=0, y_{2}=0,...,y_{n}=0|p_{l})(1-\mu_{0})}\\ & = \frac{(1-p_{h})^{n}\mu_{0}}{(1-p_{h})^{n}\mu_{0} + (1-p_{l})^{n}(1-\mu_{0})}. \end{align*}\]

Note that \(μ_n\) can be expressed recursively

\[\begin{align*} μ_n(0) = \frac{(1-p_{h})\mu_{n-1}}{(1-p_{h})^{n}\mu_{n-1} + (1-p_{l})^{n}(1-\mu_{n-1})}. \end{align*}\]

Exercise 4.4 (Job Search)

We consider a labor market where workers are either employed or searching for a job. Time is discrete and agents discount the future at rate \(r\), so that their discount factor \(\beta =1/(1+r).\) When a firm meets a job seeker, it offers her a constant wage contract and the firm-employee relationship lasts forever.

At the end of each period:

(i). Employed workers receive their wages.

(ii). Unemployed workers receive their benefits \(z=-1\) along with a job offer whose wage is sampled from the following distribution

\[\begin{equation*} w= \left\{ \begin{array}{l} 1\text{ with probability } p, \\ 0\text{ with probability } 1-p.% \end{array} \right. \end{equation*}\]

1. Write down the Bellman equation for an unemployed worker and for a worker employed at wage \(w\). Reinsert the value function \(E(w)\) of employed workers into that of unemployed workers \(U\) to express the latter as a function of \(w\).

2. Consider the following two job searching strategies. One where the worker accepts all job offers, which we denote by \(U^{A}\); and one where the worker rejects low-wage offers (\(w=0\)), which we denote by \(U^{R}.\) Compute the values of \(U^{A}\) and \(U^{R}.\)

3. Under which condition on \(p\) is it optimal for workers to reject low-wage offers? Provide an economic intuition for your result. (Hint: Remember that \(z=-1.\))

4. We now assume that the worker search for only two periods. What is the value function \(U_{2}\) of an unemployed worker in the second and last search period?

5. Consider now the value of being unemployed in the first search period \(U_{1}\). Again, compare two job searching strategies: one where the worker accepts all job offers (\(U_{1}^{A}\)); and one where the worker rejects low-wage offers (\(U_{1}^{R}\)). Compute the two value functions.

6. Under which condition is it optimal to reject low-wage offers in the first period? Compare your answer to the solution in 3. Provide an interpretation for your finding.

7. We now assume that workers are uncertain about the probability \(p\) at which they receive high-wage offers. Their initial belief at the beginning of period 1 is given by

\[\begin{equation*} p=\left\{ \begin{array}{l} p_{h}\text{ with probability } \mu _{1}, \\ p_{l}\text{ with probability } 1-\mu _{1}, \end{array} \right. \end{equation*}\]

where \(p_{h}>p_{l}.\) Use Bayes’ rule to express the agent’s posterior \(\mu _{2}(0)\) at the beginning of period \(2\) after she has received a low wage-offer. Show that \(\mu _{2}(0)<\mu _{1}.\) In particular, what is the value of \(\mu_{2}(0)\) when \(p_{h}=1?\)

8. We assume that \(p_{h}=1\). Use Bayes’ rule to compute \(U_{1}^{R}\) and \(U_{1}^{A}\). Under which condition is it optimal to reject a low-wage offer in period 1?

Solution

1. The Bellman equation for an unemployed worker can be written as:

\[\begin{align*} U = \frac{1}{1+r}(z + p*\text{max}\{U, E(\overline{w})\} + (1-p)*\text{max}\{U, E(\underline{w})\}), \end{align*}\]

where \(\overline{w} = 1\) and \(\underline{w} = 0\). Reinserting the value function \(E(w) = \frac{w}{r}\) into \(U\), we find that

\[\begin{equation*} U = \frac{1}{1+r}\left(z + p*\text{max}\left\{U, \frac{\overline{w}}{r}\right\} +(1-p)* \text{max}\left\{U, \frac{\underline{w}}{r}\right\}\right). \end{equation*}\]

2. The value of the job search srategy that accepts all job offer is given by:

\[\begin{align*} U^A = \frac{1}{1+r}\left(z + p\frac{\overline{w}}{r} + (1-p)\frac{\underline{w}}{r}\right) = \frac{p-r}{(1+r)r},\\ \end{align*}\]

where the equality follows reinserting \(\overline{w}=1\), \(\underline{w}=0\), and \(z=-1\). Similarly, the value of the job search strategy that rejects low-wage offers is given by:

\[\begin{align*} & U^R = \frac{1}{1+r}\left(z + p\frac{\overline{w}}{r} + (1-p)U^R\right) \Rightarrow U^R = \frac{p-r}{(p +r)r}. \end{align*}\]

3. Since the denominator of \(U^R\) is smaller than the denominator of \(U^A\), \(U^R\) is greater than \(UÂ\) whenever their numerators are positive, that is when \(p>r\). Intuitively, the probability of receiving a high-wage offer must be high enough to compensate for the opportunity cost \(r\) of remaining unemployed.

4. In the second period, if the worker rejects the offer, she will stay unemployed forever and receive \(\frac{z}{r}<0=\frac{\underline{w}}{r}\). Therefore, workers will accept all offers in the second period so that \(U_2 = U^A = \frac{p-r}{(1 +r)r}\).

5. \(U_1^A\) and \(U_1^R\) can be expressed as:

\[\begin{align*} & U_1^A = \frac{p-r}{(1+r)r},\\ & U_1^R = \frac{1}{1+r}\left(z + \frac{p}{r} + (1-p)U_2\right) = U_1^A + \frac{1-p}{1+r}U_2. \end{align*}\]

6. For \(U_1^R\) to be greater than \(U_1^A\), the value of searching in the second and last period \(U_2>0\) must be positive, which holds true whenever \(p>r\). This is the same condition as in the infinite horizon problem derived in 3.

7. According to Bayes’ rule, \(\mu _{2}(0)\) can be expressed as:

\[\begin{align*} \mu _{2}(0) &= Pr(p=p_h| w_1=0)\\ &= \frac{Pr(w_1=0|p_h)Pr(p=p_h)}{Pr(w_1=0|p_h)Pr(p=p_h)+Pr(w_1=0|p_l)Pr(p=p_l)}\\ &= \frac{(1-p_h)\mu_1}{(1-p_h)\mu_1+(1-p_l)(1-\mu_1)}. \end{align*}\]

The condition \(\mu _{2}(0)<\mu _{1}\) is equivalent to \(1/\mu _1<1/\mu _{2}(0)\). The second condition holds true whenever

\[\begin{align*} \frac{1}{\mu_1} &< \frac{(1-p_h)\mu_1+(1-p_l)(1-\mu_1)}{(1-p_h)\mu_1}\\ \Leftrightarrow 1-p_h &< (1-p_h)\mu_1+(1-p_l)(1-\mu_1)\\ \Leftrightarrow 1-p_h &< 1-p_l, \end{align*}\]

which holds true by definition. As expected, workers put less weight on the optimistic prior \(p=p_h\) after having received a low-wage offer. In particular, when \(p_h=1\), \(\mu _{2}(0) = 0\) because workers can exclude the belief that they will always receive a high-wage offer.

8. With \(p_h=1\), \( U_1^A\) and \(U_1^R\) can be expressed as:

\[\begin{align*} &U_1^A = \frac{1}{1+r}\left(z + \frac{\mu_1}{r} + (1-\mu_1)\frac{p_l}{r}\right),\\ &U_1^R = \frac{1}{1+r}\left(z + \frac{\mu_1}{r} + (1-\mu_1)\frac{p_l}{r}+(1-\mu_1)(1-p_l)U_2(0)\right), \end{align*}\]

where, according to the answer to question 4, \(U_2(0) = U^A(0)=\frac{1}{1+r}(z + \frac{\mu_2(0)}{r} + (1-\mu_2(0))\frac{p_l}{r})\). Since, \(\mu_2(0)=0\) when \(p_h=1\), we have \(U_2(0)=\frac{p_l-r}{(1+r)r}\). Therefore, we find that

\[\begin{align*} U_1^R > U_1^A \Leftrightarrow U_2(0)=\frac{p_l-r}{(1+r)r}>0. \end{align*} \]

The worker find it profitable to reject all low-wage offers in the first period when \(p_l>r\). This condition is similar to the one in the environment without learning except that now the pessimistic belief has replaced the average one.