GReinSS (Generative Reinforcement Learning of Structured States, ICML 2026) is a modern machine learning technique for learning structured biological latent states from indirect observation data. This approach assumes there exists some ground truth distribution of latent states S together with a probabilistic process of generating observations X given latent states S. GReinSS trains a generative policy of latent states S, with parameters θ, in order to maximize the probability of the set of observations X. In other words, it learns an accurate generative model of latent states only through indirect measurement data without any access to ground truth latent states.

Typical unsupervised machine learning approaches produce arbitrary continuous latent representations capable of reconstructing input data. In contrast, many biological problems have unobserved ground truth biological variables in the form of discrete structured latent states. Such biological latent states could consist of evolutionary trees, copy number variants, RNA isoforms present in a sample, etc. Rather than learning continuous latent representations and hoping there exists a connection with ground truth biology, we define our latent states directly as biological states of interest.

Observation data consists of the biological measurements that provide partial information about the underlying latent states. This can consist of genomics, transcriptomics, mass spectrometry, etc. We model these observations X as probabilistically coming from the underlying latent states S. For example, sequencing data X is probabilistically generated by the underlying biological state S of the sample being sequenced.

Reinforcement learning provides a natural framework for learning generative policies over complex latent state spaces. However, to discover latent states one must learn a distribution over states that maximizes observation data likelihood rather than maximizing the expectation value of some known reward function. To achieve this, we define dynamic rewards such that policy gradients result in a provably unbiased estimator of the gradient of the observation data log-likelihood. Intuitively, dynamic rewards ensure that the policy gradient is always in the direction of maximizing the probability of the observation data.

The first step is determining how to formulate your measurement data as observations X and how to formulate your unobserved biological variables of interest into latent states S. Next, we can formulate a reinforcement learning policy for generating your latent states S, as well as a probability distribution for generating observations X given latent states. After this, an initial implementation can be written, and the formulation can be modified if needed to best fit your data and goals.