Academic literature on the topic 'Gittens index'

Bandit processes and the Gittins index have provided powerful and elegant theory and tools for the optimization of allocating limited resources to competitive demands. In this paper we extend the Gittins theory to more general branching bandit processes, also referred to as open bandit processes, that allow uncountable states and backward times. We establish the optimality of the Gittins index policy with uncountably many states, which is useful in such problems as dynamic scheduling with continuous random processing times. We also allow negative time durations for discounting a reward to account for the present value of the reward that was received before the present time, which we refer to as time-backward effects. This could model the situation of offering bonus rewards for completing jobs above expectation. Moreover, we discover that a common belief on the optimality of the Gittins index in the generalized bandit problem is not always true without additional conditions, and provide a counterexample. We further apply our theory of open bandit processes with time-backward effects to prove the optimality of the Gittins index in the generalized bandit problem under a sufficient condition.

Bandit processes and the Gittins index have provided powerful and elegant theory and tools for the optimization of allocating limited resources to competitive demands. In this paper we extend the Gittins theory to more general branching bandit processes, also referred to as open bandit processes, that allow uncountable states and backward times. We establish the optimality of the Gittins index policy with uncountably many states, which is useful in such problems as dynamic scheduling with continuous random processing times. We also allow negative time durations for discounting a reward to account for the present value of the reward that was received before the present time, which we refer to as time-backward effects. This could model the situation of offering bonus rewards for completing jobs above expectation. Moreover, we discover that a common belief on the optimality of the Gittins index in the generalized bandit problem is not always true without additional conditions, and provide a counterexample. We further apply our theory of open bandit processes with time-backward effects to prove the optimality of the Gittins index in the generalized bandit problem under a sufficient condition.

Nash's generalization of Gittins’ classic index result to so-called generalized bandit problems (GBPs) in which returns are dependent on the states of all arms (not only the one which is pulled) has proved important for applications. The index theory for special cases of this model in which all indices are positive is straightforward. However, this is not a natural restriction in practice. An earlier proposal for the general case did not yield satisfactory index-based suboptimality bounds for policies — a central feature of classical Gittins index theory. We develop such bounds via a notion of duality for GBPs which is of independent interest. The index which emerges naturally from this analysis is the reciprocal of the one proposed by Nash.

Nash's generalization of Gittins’ classic index result to so-called generalized bandit problems (GBPs) in which returns are dependent on the states of all arms (not only the one which is pulled) has proved important for applications. The index theory for special cases of this model in which all indices are positive is straightforward. However, this is not a natural restriction in practice. An earlier proposal for the general case did not yield satisfactory index-based suboptimality bounds for policies — a central feature of classical Gittins index theory. We develop such bounds via a notion of duality for GBPs which is of independent interest. The index which emerges naturally from this analysis is the reciprocal of the one proposed by Nash.

In 1988 Whittle introduced an important but intractable class of restless bandit problems which generalise the multiarmed bandit problems of Gittins by allowing state evolution for passive projects. Whittle's account deployed a Lagrangian relaxation of the optimisation problem to develop an index heuristic. Despite a developing body of evidence (both theoretical and empirical) which underscores the strong performance of Whittle's index policy, a continuing challenge to implementation is the need to establish that the competing projects all pass an indexability test. In this paper we employ Gittins' index theory to establish the indexability of (inter alia) general families of restless bandits which arise in problems of machine maintenance and stochastic scheduling problems with switching penalties. We also give formulae for the resulting Whittle indices. Numerical investigations testify to the outstandingly strong performance of the index heuristics concerned.

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.
Some sections in thesis unnumbered.
Includes bibliographical references.
Imagine a world where computers are able to present desired information to people in the most relevant and effective way possible, where machines are able to adapt the way they interact with humans when they encounter different personality styles. Web Morphing captures the essence of this idea and applies it to realm of Digital Marketing, allowing companies to present product information in a manner in which the consumers are most comfortable with. By using user click-history, a Website with Morphing capability can display its information based on the user's inferred Cognitive and Cultural Styles. This thesis documents the process of building the Mathematical Inference Engine of a Web Morphing System that gives a Web site the ability to adapt itself to individual users. First, I will briefly discuss the history and motivation of Morphing. Then, I will discuss the theory of Morphing from the work of Hauser, Urban, Liberali, and Braun, and I will give a system overview of the Web Morphing System. The main contribution of the thesis is the technical implementation of the Mathematical Inference Engine, and I will describe in detail the construction of Mathematical Inference Engine's two major parts: the Bayesian Inference Engine, and the Gittins' Index Engine.
by Clarence Lee.
M.Eng.

The present work deals with the development and application of an acoustic long-period fiber grating (LPG) in conjunction with a special optical fiber (SF). The acoustic LPG converts selected optical modes of the SF. Some of these modes are characterized by complex, yet cylindrically symmetric polarization and intensity patterns. Therefore, they are the guided variant of so called cylindrical vector beams (CVBs). CVBs find applications in numerous fields of fundamental and applied optics. Here, an application to high-resolution light microscopy is demonstrated. The field distribution in the tight microscope focus is controlled by the LPG, which in turn creates the necessary polarization and intensity distribution for the microscope illumination. A gold nanoparticle of 30 nm diameter is used to probe the focal field with sub-wavelength resolution. The construction and test of the acoustic LPG are discussed in detail. A key component is the piezoelectric transducer that excites flexural acoustic waves in the SF, which are the origin of an optical mode conversion. A mode conversion efficiency of 85% was realized at 785 nm optical wavelength. The efficiency is, at present, mainly limited by the spectral positions and widths of the transducer’s acoustic resonances. The SF used with the LPG separates the propagation constants of the second-order polarization modes, so they can be individually excited and are less sensitive to distortions than in standard weakly-guiding fibers. The influence of geometrical parameters of the fiber core on the propagation constant separation and on the mode fields is studied numerically using the multiple multipole method. From the simulations, a simple mode coupling scheme is developed that provides a qualitative understanding of the experimental results achieved with the LPG. The refractive index profile of the fiber core was originally developed by Ramachandran et al. However, an important step of the present work is to reduce the SF’s core size to counteract the the appearance of higher-order modes at shorter wavelengths which would otherwise spoil the mode purity. Using the acoustic LPG in combination with the SF produces a versatile device to generate CVBs and other phase structures beams. This fiber-optical method offers beam profiles of high quality and achieves good directional stability of the emitted beam. Moreover, the device design is simple and can be realized at low cost. Future developments of the acoustic LPG will aim at applications to fiber-optical sensors and optical near-field microscopy
Diese Arbeit behandelt die Entwicklung und Anwendung eines akustischen langperiodischen Fasergitters (LPG) in Verbindung mit einer optischen Spezialfaser (SF). Das akustische LPG wandelt ausgewählte optische Modi der SF um. Einige dieser Modi weisen eine komplexe, zylindersymmetrische Polarisations- und Intensitätsverteilung auf. Diese sind eine Form der so genannten zylindrischen Vektor-Strahlen (CVBs), welche in zahlreichen Gebieten der wissenschaftlichen und angewandten Optik zum Einsatz kommen. In dieser Arbeit wird eine Anwendung auf die hochauflösende Lichtmikroskopie demonstriert. Die fokale Feldverteilung wird dabei durch die Auswahl der vom LPG erzeugten Modi, welche zur Beleuchtung genutzt werden, eingestellt. Als Nachweis wird die entstehende laterale Feldverteilung mithilfe eines Goldpartikels (Durchmesser 30 Nanometer) vermessen. Aufbau und Test des akustischen LPGs werden im Detail besprochen. Eine wichtige Komponente ist ein piezoelektrischer Wandler, der akustische Biegewellen in der SF anregt. Diese sind die Ursache der Umwandlung optischer Modi. Die maximale Konversionseffizienz betrug 85% bei 785 nm (optischer) Wellenlänge. Die Effizienz ist derzeit hauptsächlich durch die Lage der akustischen Resonanzfrequenzen des Wandlers und deren Bandbreite begrenzt. Die benutzte SF spaltet die Ausbreitungskonstanten von Polarisationsmodi zweiter Ordnung auf, sodass diese individuell angeregt werden können und weniger anfällig gegen über Störungen der Faser sind, als das bei gewöhnlichen, schwach führenden Glasfasern der Fall ist. Das zu Grunde liegende Brechzahlprofil des Faserkerns wurde von Ramachandran et al. entwickelt. Für diese Arbeit wurde jedoch die Ausdehnung des Profils verkleinert – ein erster Schritt um Anwendungen bei kürzeren optischen Wellenlängen zu ermöglichen. Es werden numerische Simulationen mit der Methode der multiplen Multipole zur Berechnung der Modenfelder und den zugehörigen Propagationskonstanten vorgestellt. Diese zeigen u. a. den starken Einfluss von geometrischen Veränderungen des Faserkerns. Basierend auf den Simulationsergebnissen wird ein einfaches Kopplungsschema für die Modi entwickelt, welches ein qualitatives Verständnis der experimentellen Ergebnisse ermöglicht. In Kombination bilden die SF und das LPG ein vielseitiges Gerät zur Erzeugung von CVBs und anderen Strahlen mit komplexer Phasenstruktur. Die Methode besticht durch hohe Qualität des Strahlprofils, stabile Abstrahlrichtung, einfachen Aufbau, elektronische Steuerbarkeit und geringe Materialkosten. Zukünftige Weiterentwicklungen des akustischen LPGs zielen auf die Anwendung in faseroptischen Sensoren und in der optischen Nahfeldmikroskopie ab

In this work, the fast evaluation and reconstruction of multivariate trigonometric polynomials with frequencies supported on arbitrary index sets of finite cardinality is considered, where rank-1 lattices are used as spatial discretizations. The approximation of multivariate smooth periodic functions by trigonometric polynomials is studied, based on a one-dimensional FFT applied to function samples. The smoothness of the functions is characterized via the decay of their Fourier coefficients, and various estimates for sampling errors are shown, complemented by numerical tests for up to 25 dimensions. In addition, the special case of perturbed rank-1 lattice nodes is considered, and a fast Taylor expansion based approximation method is developed. One main contribution is the transfer of the methods to the non-periodic case. Multivariate algebraic polynomials in Chebyshev form are used as ansatz functions and rank-1 Chebyshev lattices as spatial discretizations. This strategy allows for using fast algorithms based on a one-dimensional DCT. The smoothness of a function can be characterized via the decay of its Chebyshev coefficients. From this point of view, estimates for sampling errors are shown as well as numerical tests for up to 25 dimensions. A further main contribution is the development of a high-dimensional sparse FFT method based on rank-1 lattice sampling, which allows for determining unknown frequency locations belonging to the approximately largest Fourier or Chebyshev coefficients of a function
In dieser Arbeit wird die schnelle Auswertung und Rekonstruktion multivariater trigonometrischer Polynome mit Frequenzen aus beliebigen Indexmengen endlicher Kardinalität betrachtet, wobei Rang-1-Gitter (rank-1 lattices) als Diskretisierung im Ortsbereich verwendet werden. Die Approximation multivariater glatter periodischer Funktionen durch trigonometrische Polynome wird untersucht, wobei Approximanten mittels einer eindimensionalen FFT (schnellen Fourier-Transformation) angewandt auf Funktionswerte ermittelt werden. Die Glattheit von Funktionen wird durch den Abfall ihrer Fourier-Koeffizienten charakterisiert und mehrere Abschätzungen für den Abtastfehler werden gezeigt, ergänzt durch numerische Tests für bis zu 25 Raumdimensionen. Zusätzlich wird der Spezialfall gestörter Rang-1-Gitter-Knoten betrachtet, und es wird eine schnelle Approximationsmethode basierend auf Taylorentwicklung vorgestellt. Ein wichtiger Beitrag dieser Arbeit ist die Übertragung der Methoden vom periodischen auf den nicht-periodischen Fall. Multivariate algebraische Polynome in Chebyshev-Form werden als Ansatzfunktionen verwendet und sogenannte Rang-1-Chebyshev-Gitter als Diskretisierungen im Ortsbereich. Diese Strategie ermöglicht die Verwendung schneller Algorithmen basierend auf einer eindimensionalen DCT (diskreten Kosinustransformation). Die Glattheit von Funktionen kann durch den Abfall ihrer Chebyshev-Koeffizienten charakterisiert werden. Unter diesem Gesichtspunkt werden Abschätzungen für Abtastfehler gezeigt sowie numerische Tests für bis zu 25 Raumdimensionen. Ein weiterer wichtiger Beitrag ist die Entwicklung einer Methode zur Berechnung einer hochdimensionalen dünnbesetzten FFT basierend auf Abtastwerten an Rang-1-Gittern, wobei diese Methode die Bestimmung unbekannter Frequenzen ermöglicht, welche zu den näherungsweise größten Fourier- oder Chebyshev-Koeffizienten einer Funktion gehören

We consider multivariate trigonometric polynomials with frequencies supported on a fixed but arbitrary frequency index set I, which is a finite set of integer vectors of length d. Naturally, one is interested in spatial discretizations in the d-dimensional torus such that - the sampling values of the trigonometric polynomial at the nodes of this spatial discretization uniquely determines the trigonometric polynomial, - the corresponding discrete Fourier transform is fast realizable, and - the corresponding fast Fourier transform is stable. An algorithm that computes the discrete Fourier transform and that needs a computational complexity that is bounded from above by terms that are linear in the maximum of the number of input and output data up to some logarithmic factors is called fast Fourier transform. We call the fast Fourier transform stable if the Fourier matrix of the discrete Fourier transform has a condition number near one and the fast algorithm does not corrupt this theoretical stability. We suggest to use rank-1 lattices and a generalization as spatial discretizations in order to sample multivariate trigonometric polynomials and we develop construction methods in order to determine reconstructing sampling sets, i.e., sets of sampling nodes that allow for the unique, fast, and stable reconstruction of trigonometric polynomials. The methods for determining reconstructing rank-1 lattices are component{by{component constructions, similar to the seminal methods that are developed in the field of numerical integration. During this thesis we identify a component{by{component construction of reconstructing rank-1 lattices that allows for an estimate of the number of sampling nodes M |I|\le M\le \max\left(\frac{2}{3}|I|^2,\max\{3\|\mathbf{k}\|_\infty\colon\mathbf{k}\in I\}\right) that is sufficient in order to uniquely reconstruct each multivariate trigonometric polynomial with frequencies supported on the frequency index set I. We observe that the bounds on the number M only depends on the number of frequency indices contained in I and the expansion of I, but not on the spatial dimension d. Hence, rank-1 lattices are suitable spatial discretizations in arbitrarily high dimensional problems. Furthermore, we consider a generalization of the concept of rank-1 lattices, which we call generated sets. We use a quite different approach in order to determine suitable reconstructing generated sets. The corresponding construction method is based on a continuous optimization method. Besides the theoretical considerations, we focus on the practicability of the presented algorithms and illustrate the theoretical findings by means of several examples. In addition, we investigate the approximation properties of the considered sampling schemes. We apply the results to the most important structures of frequency indices in higher dimensions, so-called hyperbolic crosses and demonstrate the approximation properties by the means of several examples that include the solution of Poisson's equation as one representative of partial differential equations.

We consider multivariate trigonometric polynomials with frequencies supported on a fixed but arbitrary frequency index set I, which is a finite set of integer vectors of length d. Naturally, one is interested in spatial discretizations in the d-dimensional torus such that - the sampling values of the trigonometric polynomial at the nodes of this spatial discretization uniquely determines the trigonometric polynomial, - the corresponding discrete Fourier transform is fast realizable, and - the corresponding fast Fourier transform is stable. An algorithm that computes the discrete Fourier transform and that needs a computational complexity that is bounded from above by terms that are linear in the maximum of the number of input and output data up to some logarithmic factors is called fast Fourier transform. We call the fast Fourier transform stable if the Fourier matrix of the discrete Fourier transform has a condition number near one and the fast algorithm does not corrupt this theoretical stability. We suggest to use rank-1 lattices and a generalization as spatial discretizations in order to sample multivariate trigonometric polynomials and we develop construction methods in order to determine reconstructing sampling sets, i.e., sets of sampling nodes that allow for the unique, fast, and stable reconstruction of trigonometric polynomials. The methods for determining reconstructing rank-1 lattices are component{by{component constructions, similar to the seminal methods that are developed in the field of numerical integration. During this thesis we identify a component{by{component construction of reconstructing rank-1 lattices that allows for an estimate of the number of sampling nodes M |I|\le M\le \max\left(\frac{2}{3}|I|^2,\max\{3\|\mathbf{k}\|_\infty\colon\mathbf{k}\in I\}\right) that is sufficient in order to uniquely reconstruct each multivariate trigonometric polynomial with frequencies supported on the frequency index set I. We observe that the bounds on the number M only depends on the number of frequency indices contained in I and the expansion of I, but not on the spatial dimension d. Hence, rank-1 lattices are suitable spatial discretizations in arbitrarily high dimensional problems. Furthermore, we consider a generalization of the concept of rank-1 lattices, which we call generated sets. We use a quite different approach in order to determine suitable reconstructing generated sets. The corresponding construction method is based on a continuous optimization method. Besides the theoretical considerations, we focus on the practicability of the presented algorithms and illustrate the theoretical findings by means of several examples. In addition, we investigate the approximation properties of the considered sampling schemes. We apply the results to the most important structures of frequency indices in higher dimensions, so-called hyperbolic crosses and demonstrate the approximation properties by the means of several examples that include the solution of Poisson's equation as one representative of partial differential equations.