Ask These 7 Questions Before You Buy Custom Software Development Services for Healthcare AI 

Before you buy custom software development services for healthcare AI, ask each vendor how it will prove the system gives correct answers to your patients, your documents and your workflows. A polished demo runs on examples the vendor picked, so it says little about how the system will handle a Monday queue of incomplete referrals. 

Moving from demo to daily use is where AI work often slows down. In Deloitte’s State of AI in the Enterprise 2026 report, only 25% of respondents had moved 40% or more of their AI pilots into production. That survey covered 3,235 business and IT leaders across 24 countries and was fielded in August and September 2025. 

Healthcare buyers now carry one more task. On 26 October 2026, ONC finalized its HTI-5 rule, which removes the AI “model card” requirements from EHR certification. Details on how a predictive tool was trained and tested now must be requested from the vendor and written into your contract. 

This checklist is for a CIO, IT procurement lead or digital health product owner at a health system, payer or healthtech firm who is running an AI vendor selection now. Each question comes with what a thin answer sounds like and what to ask for in writing. 

Key takeaways 

• Ask for proof of your own cases. A vendor should agree to build a scored evaluation set from your de-identified data, signed off by your clinicians, before the main build starts. 

• Map where patient data goes. Ask for a data-flow diagram that names every subprocessor, including any third-party model of API, and a signed business associate agreement (BAA). 

• Settle the regulatory status early. FDA’s revised clinical decision support guidance (January 2026) and ONC’s HTI-5 rule (October 2026) both change what you should ask about intended use and transparency. 

• Own what you pay for. Code, prompts, the evaluation set, and any fine-tuned model weights should transfer to you, with exit terms agreed at signing. 

• Score answers on evidence. A short scoring sheet keeps the shortlist honest when the demos look alike. 

Why does healthcare AI change what you ask a software vendor? 

Healthcare AI changes vendor questions because its output varies, and its mistakes can reach patients. A regular application either works or shows an error. An AI feature can return a fluent but wrong answer that a busy clinician accepts. So, the buyer must define correct behavior, check data handling, and confirm regulatory status before the build begins, because each of these is expensive to change once the system is live. 

Buying is also moving faster. Menlo Ventures’ 2025 State of AI in Healthcare report, based on a survey of more than 700 US healthcare executives in August and September 2025, found that health system buying cycles for AI averaged 6.6 months, down from 8.0 months for traditional IT purchases. A shorter cycle leaves less time for diligence, so each question you put to vendors of custom software development services in your request proposal (RFP) must do more work. 

Transparency rules have shifted as well. Since January 2025, certified EHR software had to give users access to “source attributes” for decision support tools, covering intended use, training data, validation, and performance. ONC’s HTI-5 final rule removed those model card requirements, and ONC has said it may revisit them later. For now, a custom build lets you make that information a named deliverable in your contract. 

Seven questions to send every custom software development services vendor 

Add these questions to your healthcare AI RFP and ask every shortlisted vendor to answer them in writing, with evidence attached: 

1. What will “correct” mean for this system, and who signs that definition? 

2. Where will patient data travel during the build, and who can see it? 

3. How will the software connect to our EHR and to the way clinicians work today? 

4. What regulatory status could this software have, and who does that work? 

5. How will we know when the system gets worse after launch? 

6. Who owns the code, the prompts, the evaluation set, and any trained model? 

7. If we end the contract, what do we hold on to the last day? 

Each question matters at a different point in the project, so they are grouped by when a weak answer starts to cost you money. 

Before you sign: can the vendor prove it works on your cases? 

Two questions decide whether a vendor can show evidence before you commit to budget. 

1. What will “correct” mean for this system, and who signs that definition? 

A credible vendor defines correct behavior with your clinical and operations staff before writing production code. That definition becomes an evaluation set: a fixed group of real, de-identified cases with the expected answer for each, scored the same way every time the system changes. Without one, nobody can measure whether a new release improved the results. 

For example, an evaluation set for a discharge summary assistant might hold a few hundred past summaries, each with the facts a reviewer expects to see, the facts that must never appear, and a pass mark agreed by your clinical lead. For a prior authorization tool, it would hold past requests with the decision and the policy line that supports it. 

This matches where healthcare guidance is heading. Guidance on the responsible use of AI from the Joint Commission and the Coalition for Health AI (CHAI), released on 17 September 2025, asks health organizations to validate AI tools locally and keep monitoring them after deployment. An evaluation set built with your team gives you the material for both. 

What a thin answer sounds like: “Our model scores 95% on a public benchmark.” Public benchmarks do not contain your patients or your forms. 

What to ask for in writing: an anonymized evaluation report from a past project, the number of cases planned for yours, who labels them, and the pass mark that gates each release. 

2. Where will patient data travel during the build, and who can see it? 

Ask for a data-flow diagram that shows every place protected health information (PHI) will sit or pass through, from development to production. PHI is any health information that can identify a patient. A good diagram names every subprocessor, including cloud hosting and any third-party model of API, and states which environments use real, de-identified or synthetic data. 

Under HIPAA, a vendor that handles PHI for you is a business associate and must sign a business associate agreement. Check that the BAA chain reaches every subprocessor the vendor uses, and that development and test environments never hold identifiable data unless you approve it. 

Rules here are also tightening. HHS proposed the first major update to the HIPAA Security Rule in more than a decade, including a duty to verify that business associates have their safeguards in place. According to Fierce Healthcare, the final rule is now listed for July 2027. Buyers in India face a nearer date: data fiduciary duties under the Digital Personal Data Protection (DPDP) Rules, 2025 take effect on 13 May 2027. 

What a thin answer sounds like: “We are HIPAA compliant.” HHS does not certify vendors as HIPAA compliant, so the claim on its own proves little. 

What to ask for in writing: the data-flow diagram, a BAA template the vendor will sign, the full subprocessor list, how fast the vendor will tell you about a breach, and the de-identification method for test data. 

In the first sprints: will it fit how care is delivered? 

Questions three and four expose problems that usually surface in the first month of a build, when changing course is still affordable. 

3. How will the software connect to our EHR and to the way clinicians work today? 

A strong answer names the integration standard, the access route and the person on your side who approves it. Most modern EHR integration runs on HL7 FHIR APIs, a standard way for health systems to exchange records, while older HL7 v2 feeds remain common for orders and results. Your vendor should say which ones your use case needs and how long sandbox access usually takes. 

Workflow fits as much as the API. If the AI output sits in a separate tab that a nurse has to open, adoption will fall whatever the accuracy. Ask where the output appears, whether it adds or removes clicks, and how a clinician corrects or rejects it. Those corrections become useful cases for the next round of evaluation. 

Vendors with real EHR and system integration experience will also ask about your interface engine, your identity provider, and who on your side requests approval from the EHR vendor. 

What a thin answer sounds like: “We integrate with all major EHRs.” 

What to ask for in writing: the FHIR resources or HL7 messages in scope, who requests EHR vendor approval, and a plan for running alongside the current workflow before cutover. 

4. What regulatory status could this software have, and who does that work? 

Regulatory status depends on intended use, so the vendor should help you write the intended-use statement early and flag where the software could become a medical device. In the US, software that supports clinical decisions falls outside FDA device rules only if it meets four criteria, including that a clinician can independently review the basis for each recommendation. 

FDA revised its clinical decision to support software guidance on 6 January 2026. It now allows enforcement discretion for some tools that give a single recommendation when only one is clinically appropriate, and it notes that time-critical tools may fail the independent-review criterion. Small wording changes in a feature spec can move a product across these lines, so ask how the vendor tracks them. 

UK buyers have their own checkpoints. NHS England’s refreshed Digital Technology Assessment Criteria (DTAC) form replaced the earlier version from 6 April 2026. A vendor building for an NHS organization should know that form and the clinical safety standards DCB0129 and DCB0160. 

A software vendor is not your regulatory counsel. A good answer separates what the vendor documents (intended use, test evidence, change logs) from what your legal or regulatory team decides. 

What a thin answer sounds like: “It’s only a support tool, so no regulation applies.” 

What to ask for in writing: a draft intended-use statement, the evidence package the vendor will produce, and how each change to model behavior is logged. 

After go-live: who keeps it accurate, and what do you own? 

Three questions protect you once the system is live, and the project team has moved on. 

5. How will we know when the system gets worse after launch? 

Ask how the vendor detects drift, meaning a gradual drop in quality as patient mix, documents or the underlying model change. A sound plan reruns the evaluation set on every release, sends a fixed sample of live outputs for clinician review on a set schedule, and alerts a named owner when scores fall below the agreed pass mark. 

Model provider changes are a real risk. Hosted large language models are updated and retired on the provider’s timetable. Ask what happens when your model version is retired: who tests the replacement, against which cases, and who approves the switch. 

Agents need extra care. Deloitte’s 2026 report found that only 21% of companies planning to deploy agentic AI report a mature model for agent governance. If your build includes an agent that takes actions across systems, ask for logged decision traces and the conditions under which it stops and hands work to a person. 

What a thin answer sounds like: “We provide 90 days of support after launch.” 

What to ask for in writing: the monitoring metrics, the review of sample size and frequency, the response time for a quality incident, and the process for model version changes. 

6. Who owns the code, the prompts, the evaluation set and any trained model? 

You should own every artifact built for you: source code, prompts, the evaluation set, data pipelines, and any fine-tuned model weights. Where the vendor reuses its own pre-existing tools, the contract should give you a permanent, royalty-free license to keep using them after the engagement ends. 

Evaluation sets are easy to overlook. They hold your clinicians’ judgment in a reusable form, and they let a future team test changes without starting over. Ask that the set, its labeling guide and its scoring scripts sit in a repository your organization controls from day one. 

Repository ownership also shapes daily work. On the CareMonitor telemedicine platform, ViitorCloud engineers worked inside the client’s core team on a codebase the client owns, with each change going through a pull-request review. 

What a thin answer sounds like: “You own the application, but our AI engine stays with us.” 

What to ask for in writing: an IP clause that lists each artifact, a disclosure of open-source licenses in use, and confirmation that cloud and model API accounts are in your organization’s name. 

7. If we end the contract, what do we hold on the last day? 

On the last day, you should hold working code, deployment scripts, documentation, the evaluation set, access credentials, and written confirmation that the vendor has returned or deleted your data. Agree for the handover period and its price at signing, when you have the most negotiating weight. 

Ask for a run book, a document that tells a new team how to deploy, monitor and roll back the system, and ask to see one from a past project. Ask also whether a different team could rebuild the production environment from your repository alone. If the vendor hesitates, part of the system likely depends on knowledge only its engineers hold. 

What a thin answer sounds like: “Our clients never leave.” 

What to ask for in writing: handover length and fee, the data deletion method and certificate, and a test rebuild from your repository before final payment. 

How do you compare vendor answers side by side? 

Score each written answer 0, 1 or 2: zero for no evidence, one for a described process, two for an artifact you can inspect, such as a sample evaluation report or a signed BAA template. A vendor that scores zero on correctness, patient data or ownership should leave the shortlist whatever its total. Use the same sheet for every firm offering custom software development services, so every vendor is judged on the same evidence. This scoring method is an editorial framework for this article, not an industry standard. 

Question	Evidence that earns a 2	Red flag
Correct behavior	Sample evaluation report with pass marks	Public benchmark scores only
Patient data	Data-flow diagram, BAA template, subprocessor list	“HIPAA compliant” with no documents
EHR integration and workflow	Named FHIR resources and an approval plan	“We integrate with every EHR”
Regulatory status	Draft intended-use statement	“No regulation applies”
Quality after launch	Monitoring metrics and a model-change process	Support ends at launch
Ownership	IP clause listing each artifact	Vendor keeps the “AI engine”
Exit	Handover terms and a test rebuild	Exit never discussed

Run the scoring with at least one clinician and one security reviewer in the room. Where their scores differ, you have found the question that needs a follow-up call before AI vendor selection closes. 

What should a good vendor ask you in return? 

A vendor that has delivered healthcare AI before will ask about your side of the work before it quotes. Expect questions like these: 

• Clinical time: which clinicians will review outputs, and the hours a week they can give during the build. 

• Data access: whether de-identified sample data can reach the team in the first two weeks. 

• Stop rule: what result would make you end the project, and who makes that call. 

• Ownership after launch: which team runs the system day to day, and whether there is a budget for model usage and monitoring. 

If no vendor on your list asks for these, the proposals are probably priced on assumptions. Having your answers ready also shortens the first weeks of the build, since the team can start on real cases straight away. 

When is custom software the wrong purchase for healthcare AI? 

Custom software is the wrong purchase when a proven product already handles your use case inside your EHR, and your workflow looks much like other organizations’. Health systems in the Menlo Ventures survey favored tools they could deploy without heavy custom development, and for common needs such as ambient notetaking, that choice often makes sense. 

Custom healthcare AI development earns its cost in other situations. Your workflow may be specific to your organization, your data may not be allowed to leave your environment, the AI may need to combine data from more than one internal system, or you may want to own the model and evaluation set as long-term assets. 

A useful test is to ask each provider of custom software development services which off-the-shelf options it considered for your use case and why it rejected them. If a vendor proposes a full build without that comparison, read the proposal with extra care. 

How ViitorCloud approaches healthcare AI builds 

ViitorCloud’s healthcare work covers patient-facing platforms, EHR and HL7/FHIR integration, and custom AI solutions such as clinical workflow assistants. On AI projects, the team starts by agreeing with the client what correct output means and builds the evaluation set with the client’s reviewers before the main build. 

For organizations comparing custom software development services, engineers can work as a dedicated team or inside your existing team, and the code, prompts and evaluation set are delivered into repositories you control. 

Which of these questions would your current shortlist fail? 

Send all seven questions to every vendor offering custom software development services for your healthcare AI project and ask for written answers with evidence attached. Each vendor’s weakest answer is a good guide to where its project would struggle first. 

If you want a second view of the answers or want to see how ViitorCloud would answer them for your use case, talk to our healthcare AI team. We can start with a sample evaluation set built on a small slice of your de-identified cases.