Using AI to Improve Research Data Discovery

TL;DR

Use AI to help researchers find datasets, protocols and notes by meaning, not only by keyword. Every answer should link to the source record. Keep a reviewer in the loop for any output that feeds a report, a dataset or a participant decision. Test the tool on your own records before you rely on it.

Keyword search fails when a researcher does not know the exact term. A dataset called “wave 2 cohort export” may be the one a researcher needs, even when the search is “follow-up participants.” AI search can connect the two. It is useful only when its answers are traceable.

What AI search does well

AI search converts text into a form that captures meaning. It can match a question to a record that uses different words. It can also group similar records, such as protocols for the same instrument or consent forms with the same terms.

What AI search should not do alone

AI can produce an answer that sounds right and is not. Do not let a generated answer stand in for the source record. Show the record next to each answer, and let the researcher open it.

A worked example of a discovery test

Imagine a lab that tests search on 40 records. It writes 10 questions a researcher might ask, and each question has one correct record. Suppose the tool returns the correct record in the top three results for 7 of the 10 questions. That is 70 percent for this sample. The three misses point to descriptions that need better fields. The next test uses the same 10 questions after the descriptions are improved.

This is an illustrative example, not a measured result. A set of 10 questions is small, so use a first test like this as a check, then expand the set before you make decisions.

Set up review and evaluation

Agree which outputs need a reviewer. Summaries used internally may need a light review. Anything that feeds a report or a participant decision needs a named reviewer. Track errors after launch, and record each one with its cause.

Choose the model with your data rules

Decide what data may leave your environment before you choose a model. Hosted providers and self-hosted models have different data handling. Write the rules down and check them with your governance team.

Prepare the records before the model

AI search is only as good as the records it reads. Before you connect a model, check that each record has a title, an owner, a study and a date. Fill the gaps that matter most for search. Remove duplicate records, and mark superseded versions clearly. A short cleanup often improves results more than a change of model.

Write a short description for each dataset in plain language. Researchers search with the words they use, so the description should include the terms your field uses for the same thing.

Handle sensitive records with care

Some research records include personal or clinical information. Decide which records the search may index, and which must stay out. Apply the same access rules to search results that apply to the records. Test this with accounts that have limited access, and check that restricted records never appear in results.

Log the questions people ask and the records the tool returns. The log helps you find gaps in the descriptions and spot misuse. Store the log under the same access rules as the records.

Keep a person in the loop

Decide which outputs need a reviewer before launch, not after an error. Summaries for internal planning may need a light check. Outputs used in reports, data releases or participant decisions need a named reviewer who signs off. Record each review decision, so you can explain how an output was approved.

Keep improving the descriptions

Search quality depends on the descriptions that people write, and those descriptions drift over time. Review the questions that found no useful record each month. Each miss points to a missing term or an unclear description. Fix the description, not the model, and the next search improves without a new build.

FAQ

Does AI search replace a data catalogue?

No. Search helps people find records. A catalogue holds the record, owner, version and access rules. The two work well together.

How do we measure whether it works?

Write a set of real questions, mark the correct record for each, and measure how often the right record appears near the top. Repeat the test after each change.

Can the tool search records people are not allowed to see?

It should not. Apply the same access rules to search that apply to the records themselves, and test that they hold.

Next step

Our AI for research page describes how we scope a first use case. To discuss a discovery test on your own records, book a discovery call.