糖心视频

This week, the STOR-i CDT put on a Master Class for the MRes students. This is set up so that an expert in a particular subject visits 糖心视频 to give us an intense week of lectures and labs in their field. This week, the guest expert was , who is a STOR-i student working with Barry Nelson.

Another topic Prof. Nelson spoke about was Simulation Optimisation. This is a useful tool for when the objective function, $\theta(\mathbf{x})$, or constraint of an optimisation problem need to be estimated for each scenario. The estimator depends on the run length, number of replications and the pseudo-random numbers used in the simulation. Unless each scenario can be tested as many times as necessary to get $\theta(\mathbf{x})$ to a certain confidence level, the same problem that occurs for multi-armed bandit problem occurs. I discussed exploration versus exploitation in the "Why Not Let Bandits in to Help in Clinical Trials?" blog.

This is a "hot" (Nelson's words) topic in research with many different lines of approach being tried. One he described was called the Gradient Search. The idea is to use the gradient of $\theta(\mathbf{x})$ to move from the current scenario $\mathbf{x}$ to a better one. As we only have an estimate for $\theta(\mathbf{x})$, this isn't that simple. One method, known as Infinitesimal Perturbation Analysis (IPA), really struck me. It only holds in cases where $\theta(\mathbf{x})=E[Y]$, where $Y$ is the output of the simulation and under the condition that $$\frac{\partial E[Y(\mathbf{x})]}{\partial x_i}=E\left[\frac{\partial Y(\mathbf{x})}{\partial x_i}\right]$$ This is often difficult to prove for a particular simulation, but gives excellent results when it holds. Other results are normally more practical and have easier conditions to meet, but this has a wonderful mathematical cleanliness.

There were a number of other very interesting topics we were introduced to this week, and it was a brilliant opportunity to have Prof. Nelson to introduce us to them.