Using AI to Improve Research Data Discovery

TL;DR

Use AI to help researchers find datasets, protocols and notes by meaning, not only by keyword. Every answer should link to the source record. Keep a reviewer in the loop for any output that feeds a report, a dataset or a participant decision. Test the tool on your own records before you rely on it.

Keyword search fails when a researcher does not know the exact term. A dataset called “wave 2 cohort export” may be the one a researcher needs, even when the search is “follow-up participants.” AI search can connect the two. It is useful only when its answers are traceable.

What AI search does well

AI search converts text into a form that captures meaning. It can match a question to a record that uses different words. It can also group similar records, such as protocols for the same instrument or consent forms with the same terms.

What AI search should not do alone

AI can produce an answer that sounds right and is not. Do not let a generated answer stand in for the source record. Show the record next to each answer, and let the researcher open it.

A worked example of a discovery test

Imagine a lab that tests search on 40 records. It writes 10 questions a researcher might ask, and each question has one correct record. Suppose the tool returns the correct record in the top three results for 7 of the 10 questions. That is 70 percent for this sample. The three misses point to descriptions that need better fields. The next test uses the same 10 questions after the descriptions are improved.

This is an illustrative example, not a measured result. A set of 10 questions is small, so use a first test like this as a check, then expand the set before you make decisions.

Set up review and evaluation

Agree which outputs need a reviewer. Summaries used internally may need a light review. Anything that feeds a report or a participant decision needs a named reviewer. Track errors after launch, and record each one with its cause.

Choose the model with your data rules

Decide what data may leave your environment before you choose a model. Hosted providers and self-hosted models have different data handling. Write the rules down and check them with your governance team.

Prepare the records before the model

AI search is only as good as the records it reads. Before you connect a model, check that each record has a title, an owner, a study and a date. Fill the gaps that matter most for search. Remove duplicate records, and mark superseded versions clearly. A short cleanup often improves results more than a change of model.

Write a short description for each dataset in plain language. Researchers search with the words they use, so the description should include the terms your field uses for the same thing.

Handle sensitive records with care

Some research records include personal or clinical information. Decide which records the search may index, and which must stay out. Apply the same access rules to search results that apply to the records. Test this with accounts that have limited access, and check that restricted records never appear in results.

Log the questions people ask and the records the tool returns. The log helps you find gaps in the descriptions and spot misuse. Store the log under the same access rules as the records.

Keep a person in the loop

Decide which outputs need a reviewer before launch, not after an error. Summaries for internal planning may need a light check. Outputs used in reports, data releases or participant decisions need a named reviewer who signs off. Record each review decision, so you can explain how an output was approved.

Keep improving the descriptions

Search quality depends on the descriptions that people write, and those descriptions drift over time. Review the questions that found no useful record each month. Each miss points to a missing term or an unclear description. Fix the description, not the model, and the next search improves without a new build.

FAQ

Does AI search replace a data catalogue?

No. Search helps people find records. A catalogue holds the record, owner, version and access rules. The two work well together.

How do we measure whether it works?

Write a set of real questions, mark the correct record for each, and measure how often the right record appears near the top. Repeat the test after each change.

Can the tool search records people are not allowed to see?

It should not. Apply the same access rules to search that apply to the records themselves, and test that they hold.

Next step

Our AI for research page describes how we scope a first use case. To discuss a discovery test on your own records, book a discovery call.

Why Research Institutes Need Long-Term Software Engineering Partners

TL;DR

Research institutes need a long-term software partner for the life of the system, not only the first build. Grants end, staff move on, and libraries and servers need updates. A long-term software partner who documents the system, supports it and plans its handover keeps research running. Ask for a written support plan before you sign.

Teams often build research software for one grant. The grant pays for the build, and then the project ends. The system must still run for the people who use it. Without an owner, bugs go unfixed and security updates stop.

Why research software needs ongoing care

Software changes around the research. Operating systems update, vendors retire libraries and browsers change. A system that works today may fail after a quiet update. Ongoing care catches these changes before users notice.

What a long-term software partner should provide

A long-term partner should document the system as the team builds it, keep the documentation current, and apply updates on a fixed schedule. It should also track open issues, respond within agreed times and report what changed each month. Ask for these in writing.

Here is a small example. An institute runs 3 research systems. Each system has 2 kinds of work: updates and fixes. A monthly review covers all 3 systems, so the institute gets 3 system reports each month. Each report lists updates applied, fixes closed and open issues. If a report shows 2 open issues older than 30 days, the review decides their priority.

The handover question

Plan the handover before you need it. Agree what documents, access and training the institute receives when the partner steps back. A handover is smooth when the runbooks already exist and the team has used them.

What to check before you sign

Ask four questions. Who owns the code and the data? What does the support plan cover? Who requests and approves changes? How will the partner hand over the system? Get the answers in writing.

What a healthy support relationship looks like

A healthy support relationship has a named contact on each side, a shared list of open issues and a regular review. Both sides know who decides on priorities. The partner takes change requests in writing, estimates them before work starts and confirms them when the work ends. The partner reports problems early and says so when it cannot meet an agreed time.

Watch for signs of trouble. Issues that stay open for months, changes that arrive without an estimate and documentation that falls behind are early warnings. Raise them in the monthly review, not at the end of the contract.

Questions to ask a long-term software partner in the first call

Ask how the partner documents its work and where the documents live. Find out who on the partner’s team would work on your system, and what happens if that person leaves. Check how the partner tests updates before they reach your system, and how it reports a security problem. Request a description of what a handover looks like and how long it usually takes.

The answers show how the partner works. A partner that answers clearly, in writing, is easier to hold to a plan. A partner that avoids these questions will be harder to work with over several years.

Budget for the long term

Funders often set research budgets grant by grant. A support plan can sit in the institute’s core budget, so the system keeps running between grants. Show the cost of a missed update or a failed backup next to the cost of a support plan. That comparison usually makes the case without a long argument.

Plan the end of a grant before it starts

Every grant-funded system has an end date. Plan that date when you scope the system. Decide who will own the system after the grant, how the institute will fund it and who will hold the documentation. A system with a clear plan for its next owner survives the change of staff that grants often bring.

Keep the knowledge in the institute

A partner holds knowledge about your system, but the institute must hold it too. Ask the partner to share its runbooks, its test records and its change history with a named person on your side. Review this material once a year. If the partner leaves, your team can continue the work without starting over. The Software Sustainability Institute shares guidance on sustaining research software that helps you judge a long-term software partner.

FAQ

Is a long-term partner only for large institutes?

No. A small lab with one important system can benefit from a written support plan as much as a large center.

What does a support plan cost?

The price of support depends on the systems covered, the response times and the expected number of changes. We agree the scope in writing before the plan starts.

Can we bring in a different partner later?

Yes, if the documentation is current and the handover is planned. Plan for it from the start.

Next step

Read our research software maintenance page to see how a support plan is scoped. To discuss a plan for your systems, book a discovery call.