Extracting Target-Adverse Event Relationships from Scientific Literature: A Multi-Model Active Learning Approach

We presented this as a poster at Eurotox 2026, Vienna, Austria on September 13, 2026. We want to make sure this work gets published more widely.

Background and Motivation

Understanding the safety risks associated with therapeutic target modulation is a critical challenge in preclinical safety assessment. Scientific literature contains a wealth of evidence linking drug targets to phenotypic outcomes, yet the volume and complexity of publications makes literature review by toxicologists intractable at scale. We present a cost-effective machine learning approach to create an evidence dataset from literature that permits systematic assessment by toxicologists.

Recent Large Language Models (LLMs) make it easy to run natural language tasks, but they present two barriers for toxicity extraction at scale:

First, general-purpose LLM outputs are difficult to audit: conclusions are returned without consistent, traceable links to the underlying passages. They are sensitive to prompt and model versions, and can be confidently wrong — all of which limit their use in safety-critical decision making.

Second, applying LLMs to millions of full-text articles, refreshed continuously, is prohibitively expensive for most organisations.

We therefore decompose the problem into parts that can be independently verified: Several small language models each solve one narrow sub-problem at a time. Their outputs are combined into a directional target–phenotype relationship that resolves both the direction of target modulation and the direction of effect for the phenotype. Additionally, paragraphs are simplified into short snippets to make them easier to read and comprehend by humans and models alike.

Here we describe the data processing, model training, benchmarking and inference strategies implemented in our literature pipeline and a hybrid approach that takes advantage of the cost-effectiveness of smaller models with the sophistication of larger language models. This approach enables weekly refreshed evidence updates from the scientific literature.

Methods

Literature Corpus

Over 40 million articles (From Pubmed Abstracts and Pubmed Central (PMC)) were processed with a proprietary Named Entity Recognition (NER) system, based on a vocabulary of adverse events, targets and other tox-relevant entities.

Small Language Models

As the task of understanding literature is complex, the task was composed into three parts requiring three separate small language models:

• Target Modulation: Is the target up or down regulated?

• Phenotype Modulation: Is the phenotype or adverse event increasing or decreasing?

• Causality Detection: Is Target Modulation causing Phenotype Modulation?

These three models were trained using both rule-based and LLM assisted training data, fine tuning a small 300 Million parameter language model (dmislab/biobert [1]) (Figure 1)

Model Training

Initial training was carried out with rule-based data, followed by further training with LLM-annotated labels, based on uncertainty sampling with the initial model. An active training loop was repeated until model performance converged.

Figure 1: Three separate small language models identify the modulation of the target, phenotype and causal association. These are used to assemble a literature database that can be queried in various ways.

Hybrid Approach

While small language models cannot match the accuracy of frontier large language models, they are quicker and cheaper to run, so are sufficient for many cases of simple association or no relation. Their output can be used to identify the cases where large language model sophistication is required.

Based on benchmarks and application evaluation, the extraction pipeline was augmented with an LLM refinement step that resolves direction of effect in more complex cases.

Figure 2: If the output of the small language models is determined to be complex (usually based on nuanced scientific language), a large language model refines the decision.

Results

The output of the relationship model was a 11 class prediction; no relation, association and directional relationship with nine combinations of [Inhibition, Activation, Modulation] (for targets) [Increases, Decreases, Modulates] (for phenotypes). This is achieved either by combining the results of the three smaller language models or with direct prediction with the larger foundation models.

Relationship benchmark

A dedicated benchmark of 300 examples was developed to measure the performance of the small language models and compare against LLMs. The samples were annotated by experts following annotation guidelines.

Figure 3: Comparison of the small language models (green) and various large language models (orange) using the relationship benchmark on accuracy and F1 Macro Score [2]

As expected, a small language model with 300M parameters wasn’t a match for the performance of commercial LLMs with billions of parameters. This model wasn’t intended to be competitive - rather it was built to handle the majority of simpler scenarios. What is interesting is that the open weight GPT OSS 120b model produces almost exactly the same result as Claude, ChatGPT and Gemini.

Mixing Small and Large Language Models

We observed the small language model was pretty good on the No relation and Associated labels but struggled with detecting the correct modulation and effect (Figure 4). This led us to a hybrid approach, using the GPT OSS 120b model to process all directional predictions (10% of the total corpus). Figure 3 shows the effect this change had on the benchmark result, approaching that of the larger LLMs.

Figure 4: Confusion matrices show the actual versus predicted results on the benchmark, showing how different models confused different prediction classes.

Discussion

Costs of Corpus Processing

While LLMs achieve a higher accuracy, it is not feasible to use them for a full run of 300M relationship predictions. The following cost-accuracy frontier shows the cost of additional accuracy for each model over the in-house small language model baseline.

Figure 5: Cost-accuracy plot of corpus scale prediction with various language models, showing the cost of achieving additional accuracy over the baseline model.

The hybrid approach allows for comparable accuracy without incurring unmanageable cost. It is worth noting that the commercial LLMs do not provide significant additional benefit over mid-size models (GPT OSS), despite being much more costly.

Examples

The following shows a few examples of scientific text with their identified directional relationships.

“DGAT1-null mice have a favorable phenotype that includes decreased adiposity, increased energy expenditure, and resistance to diet-induced obesity.” -> InhibitionDecreases

“Transgenic overexpression of DGAT1 in mouse skeletal muscle protected mice against high-fat diet-induced insulin resistance.” -> ActivationDecreases

“The overexpression of DGAT1 in the heart resulted in the development of cardiomyopathy, an increase in cardiac fibrosis, and a decrease in heart mitochondrial biogenesis over time.” -> ActivationIncreases

Conclusions

Scientific literature contains a significant number of important observations and relationships and allows for the inclusion of important safety signals in addition to experimental, mechanistic and clinical insights.

Through iterative development of data processing pipeline and proprietary ML models, we are able to update literature data weekly with new findings, conferring the advantage of large language model accuracy without the associated compute and environmental costs.

References

[1] BioBERT: a pre-trained biomedical language representation model for biomedical text mining, Lee et al, Bioinformatics, Volume 36, Issue 4, February 2020, Pages 1234–1240

[2] F1 macro is the unweighted average of per-class F1 scores (the harmonic mean of precision and recall for each class), giving equal weight to rare and common categories rather than favoring classes with more examples.