What is the AI Opportunity Assessment?

The AI Opportunity Assessment is a fixed-scope diagnostic from AdwumaTech that evaluates an organization's data, workflows, and systems and returns a ranked, costed, sequenced AI plan the client owns and can take to any vendor. It maps to the MAP function of the NIST AI Risk Management Framework and is the entry point to AdwumaTech's enterprise practice.

Should you start with the highest-value AI use case?

No. AdwumaTech treats this as the most common sequencing error. High value processes are usually complex, deeply integrated, and consequential when wrong. The correct first build is the shortest deployment distance that produces measurable recovery.

Why do AI pilots fail to reach production?

AI pilots fail when curated pilot performance stops predicting production performance. The usual causes are data that was not ready, success criteria that were never measured, production-scale cost, and no owner when the system misbehaves.

What is deployment distance?

Deployment distance is the engineering work between a model that performs on a benchmark and a system that runs in production under real inputs, real volume, and real consequences. AdwumaTech scores it across data readiness, workflow determinacy, integration depth, failure tolerance, and oversight cost.

How do you calculate the oversight cost of an AI system?

Multiply the model's measured error rate on production-representative inputs by intended volume, then by the reviewer time required to catch and correct a wrong output. AdwumaTech expresses the result as review hours per thousand outputs and models it against a risk-coverage curve.

How does the Deployment Readiness Score relate to the ML Test Score?

AdwumaTech's Deployment Readiness Score is a pre-build triage instrument used to decide what to build. The ML Test Score is a pre-launch verification instrument used to decide whether what was built is ready to run.

Can an AI use case score well and still fail?

Yes. A high Deployment Readiness Score reduces engineering risk, but it does not remove organizational risk. A use case with no accountable owner for its output can still fail at adoption.

Who owns what AdwumaTech builds?

The client. For enterprise engagements the models, weights, pipelines, and code belong to the client without restriction. The assessment deliverable is owned by the client from delivery.

What security standards does AdwumaTech operate under?

AdwumaTech is ISO 27001 certified for information security management across all engagements. Data handling is GDPR aligned and documented for security and procurement review under NDA.

Which AI Use Case to Build First: AdwumaTech's Five-Axis Method

Q: What is selective prediction and why does it matter for cost?

Selective prediction allows a model to abstain on inputs where confidence falls below a threshold. Accuracy on the answered subset rises as coverage falls, which makes oversight cost a curve rather than a constant. It requires calibrated confidence.

AdwumaTech ranks enterprise AI use cases on two variables: deployment distance and recoverable cost, in that order of priority.

Deployment distance is the engineering work standing between a capable model and a system that runs inside an operation without supervision. AdwumaTech measures it across five axes: data readiness, workflow determinacy, integration depth, failure tolerance, and oversight cost. A use case carrying large annual cost and long deployment distance is a second-year project. A use case carrying moderate cost and short deployment distance is the one to build first.

Most enterprise AI portfolios are ranked on value alone. Value is the easier variable to estimate and the less predictive one. The projects that stall in production usually stall for engineering reasons visible before a line of code was written. The method below is the diagnostic AdwumaTech runs to surface those reasons while they remain cheap to act on, and it is built to be consistent with the reference frameworks the field has converged on: the NIST AI Risk Management Framework, the ML Test Score rubric, and the technical requirements of the EU AI Act.

What is the pilot ceiling?

The pilot ceiling is the point at which pilot performance stops predicting production performance. AdwumaTech uses the term to describe the most common failure pattern in enterprise AI portfolios.

The pattern is measured. RAND Corporation's root cause analysis of AI project failure found that more than 80 percent of AI projects fail, roughly twice the failure rate of non-AI technology projects. S&P Global Market Intelligence's 2025 survey found that 42 percent of companies abandoned most of their AI initiatives, up from 17 percent the year before. MIT Project NANDA's 2025 work found that around 95 percent of organizations were seeing no measurable return to the income statement from their generative AI pilots.

Three methodologies, three populations, one finding. The causes those studies name are consistent and they are not model quality: data that was not ready, success criteria never defined in measurable terms, cost that appeared only at production scale, and no owner when the system misbehaves. Each is an engineering or organizational condition a pre-build diagnostic detects.

The underlying mechanism has a technical name. Pilots are evaluated on a sample drawn from a distribution the team curated. Production draws from the operational distribution, which contains the malformed record, the customer who switches language mid-sentence, the invoice from the vendor who changed their template, and the network that drops for ninety seconds during a payment write. The gap between the two is distribution shift. It is not a model defect. It is a scoping defect that shows up as a model defect.

A pilot reaching 92 percent accuracy on curated inputs therefore establishes very little about a system that has to hold at 3am against the full input distribution. This is why organizations accumulate successful pilots and few running systems. The pilot answered a question about model capability. The unanswered question was about engineering distance.

The five axes of deployment readiness

AdwumaTech scores each candidate use case from 1 to 5 on each axis. The sum is the Deployment Readiness Score, on a scale of 5 to 25.

Axis	The question it answers	Score 1	Score 5
Data readiness	Does the data that describes this decision exist, and is it labelled to a written specification with measured inter-annotator agreement?	Data is uncollected, held in free text, or exists only in the heads of operators.	Structured, dated, retrievable through an API, labelled to specification, with schema validation and drift monitoring in place.
Workflow determinacy	Is there a correct answer, and would two qualified experts agree on it?	Judgement calls with no adjudication standard and low inter-rater reliability.	A defined correct output that experts agree on above 90 percent of the time, with an adjudication protocol for the remainder.
Integration depth	How many production systems must the output write to before value is realised?	Writes to three or more systems of record, at least one of them a core banking, ERP, or national register.	Returns a recommendation into a single interface an operator already uses.
Failure tolerance	What happens when the system is wrong, and who discovers it?	Wrong outputs are irreversible, invisible to the operator, or externally consequential.	Wrong outputs are caught in the same workflow, reversible, and cost minutes.
Oversight cost	What does human review cost per thousand outputs at the model's measured error rate on production-representative inputs?	Review cost exceeds the manual cost being displaced.	Review is sampling-based and falls as confidence thresholds tune.

A score of 20 or above is a first build. A score of 13 to 19 is a build that opens with a data or integration workstream before any model work begins. A score below 13 is a use case AdwumaTech revisits once the constraint holding it down has been removed by something else on the roadmap.

The scoring is deliberately coarse. Its purpose is to make the binding constraint visible and to force the argument into the open before budget is committed.

Why integration depth carries more weight than it appears to

Integration depth looks like a plumbing variable. It is the axis where long-run cost concentrates.

Sculley and colleagues, in Hidden Technical Debt in Machine Learning Systems, established that model code is a small fraction of a deployed ML system and that the surrounding infrastructure generates the maintenance burden: glue code, pipeline jungles, undeclared consumers, and configuration debt. Their CACE principle, Changing Anything Changes Everything, describes the property that makes ML systems expensive to integrate.

A use case writing to three systems of record does not have three times the integration cost of one writing to a single interface. It has three integration surfaces, three failure modes at the boundary, three sets of undeclared consumers, and a reconciliation requirement none of them own. AdwumaTech scores integration depth early because it is the axis that is cheapest to assess and most expensive to discover late.

Oversight cost is the axis that gets skipped

Oversight cost is the axis most portfolios omit, and the one that most often decides whether a deployed system survives its first quarter.

The arithmetic is simple and it is rarely done. Take the model's error rate on your inputs, not on a public benchmark. Multiply by the volume you intend to run. Multiply by the time a qualified reviewer takes to catch and correct a wrong output. That is the standing labour cost of the system.

On an internal AdwumaTech evaluation set of Akan and Ewe task outputs, a general purpose frontier model returned a task error rate of 38.2 percent. At the review rates a qualified native-speaker reviewer sustains, that error rate generates roughly 74 hours of human review per thousand outputs. The model is capable. The system is not economic at that error rate until the errors are reduced through post-training on domain data, or the review is designed into the workflow at a volume the operation can carry, or both.

Oversight cost is not fixed at the model's headline error rate. A system with calibrated confidence scores can abstain. Under selective prediction, the system answers where confidence exceeds a threshold and defers the remainder to a human, and accuracy on the answered subset rises as coverage falls. The operating question is not "what is the error rate" but "at what coverage does the error rate reach the level this workflow tolerates, and what does the deferred volume cost."

A risk-coverage curve answers that. A single accuracy number does not. Calibration is a prerequisite, because a model that is confidently wrong cannot abstain usefully.

How the five axes map to established frameworks

AdwumaTech built the scoring to be legible to teams already working inside recognised frameworks, and to produce artifacts those frameworks require.

Axis	NIST AI RMF 1.0 function	ML Test Score category	EU AI Act requirement if the system is high-risk
Data readiness	MAP, MEASURE	Tests for features and data	Article 10, data and data governance
Workflow determinacy	MAP	Tests for model development	Article 15, accuracy
Integration depth	MANAGE	Tests for ML infrastructure	Article 15, robustness
Failure tolerance	MEASURE, MANAGE	Monitoring tests for ML	Article 9 risk management, Article 73 serious incident reporting
Oversight cost	MANAGE	Monitoring tests for ML	Article 14, human oversight

The Deployment Readiness Score is a pre-build triage instrument. The ML Test Score is a pre-launch verification instrument. AdwumaTech uses the first to decide what to build and the second to decide whether what was built is ready to run.

Validating a score before the build commits

A score is a hypothesis about deployment distance. AdwumaTech validates it with the cheapest evidence that can falsify it, in this order.

Evaluate on a production-representative holdout sampled from live traffic across the full input distribution, including malformed and out-of-scope cases.
Build the evaluation harness before the model work. If the success criterion cannot be computed automatically, the project has no feedback signal.
Run shadow mode. The system processes live inputs and writes to a log instead of a system of record.
Release behind a canary with a defined rollback trigger, expanding coverage as the risk-coverage curve holds.
Instrument drift monitoring on inputs and outputs from day one.

Steps three and five are where the pilot ceiling is actually broken. A shadow deployment converts an argument about readiness into a number.

Recoverable cost, and how AdwumaTech prices it

Deployment readiness ranks feasibility. Recoverable cost ranks value. Both are required and only one is usually estimated.

Recoverable cost is the portion of the current cost of a process that a working system actually removes. It is smaller than total process cost, because some of that cost is exception handling, relationship work, and judgement that stays human.

AdwumaTech prices it in three components.

Direct labour recovered: hours currently spent on the handled path, at loaded cost.
Error cost recovered: current error rate multiplied by volume and the average cost of an error.
Latency cost recovered: the cost of delay between when a decision could be made and when it currently is.

Against that, the total cost of ownership includes inference cost at production volume, standing oversight cost, evaluation and monitoring infrastructure, retraining cycles, and integration maintenance. A recoverable cost figure that omits oversight and maintenance is a gross number presented as a net one.

AdwumaTech priced seven of these bottlenecks in detail in The Operational Cost of Manual Work in Ghana and West Africa. That analysis introduced the handoff tax: the compounding cost of work passing between systems and people that no single team owns.

What disqualifies a use case

AdwumaTech treats four conditions as disqualifying regardless of score.

The decision has no owner. If no named person is accountable for the output of the current process, no one will be accountable for the output of the system.

The correct answer is contested. If two qualified experts disagree on the right output more than one time in ten, there is no target for the system to hit and no basis for evaluation.

The data describing the decision was never captured. A process that has run on institutional knowledge for fifteen years carries no training signal. Instrumentation comes first, and it takes a quarter.

The consequence of a wrong output is external and irreversible. A wrong output reaching a citizen, a regulator, or a customer account with no interception point is not a first build.

The AI Opportunity Assessment

The AI Opportunity Assessment is the fixed-scope diagnostic AdwumaTech runs this method inside, and it is the first step in every AdwumaTech enterprise engagement. It corresponds to the MAP function of the NIST AI Risk Management Framework, and where a candidate use case is consequential it produces the input to an AI system impact assessment.

It returns four artifacts:

A ranked inventory of candidate use cases, each scored on the five axes with the evidence behind the score.
Recoverable cost priced per use case, against total cost of ownership, with assumptions stated and adjustable.
A sequenced plan showing which builds unblock which, and where data workstreams have to land first.
Success criteria defined per use case in terms measurable in production, with the evaluation method specified.

The deliverable belongs to the client and can be taken to any vendor. AdwumaTech builds what the assessment recommends because the success criteria defined in it are the criteria AdwumaTech is then held to through go-live.

From there an engagement takes one of two routes. AI Consulting builds the system the operation needs. Productized AI deploys one already proven and ready to run. Both stand on a capability layer AdwumaTech operates itself: data operations built in house, and NOKORE AI identity integrity where the workflow depends on it.

Bring the problem. Keep the system. Request an AI Opportunity Assessment.

References

RAND Corporation, "The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed," 2024.
S&P Global Market Intelligence, "Voice of the Enterprise: AI & Machine Learning," 2025.
MIT Project NANDA, "The GenAI Divide: State of AI in Business 2025," 2025.
D. Sculley et al., "Hidden Technical Debt in Machine Learning Systems," Advances in Neural Information Processing Systems 28, 2015.
E. Breck, S. Cai, E. Nielsen, M. Salib, D. Sculley, "The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction," IEEE International Conference on Big Data, 2017.
NIST, "Artificial Intelligence Risk Management Framework (AI RMF 1.0)," NIST AI 100-1, January 2023.
NIST, "Generative Artificial Intelligence Profile," NIST AI 600-1, July 2024.
ISO/IEC 42005:2025, Information technology, Artificial intelligence, AI system impact assessment.
Regulation (EU) 2024/1689, the Artificial Intelligence Act.

About AdwumaTech

AdwumaTech AI engineers applied AI systems for enterprises and sovereign digital infrastructure for governments, building the full chain from data acquisition through post-training to the deployed system. The enterprise practice covers AI consulting, productized AI, data operations, and NOKORE AI identity integrity. The government practice covers digital identity, payments infrastructure, citizen services, and AI assurance. AdwumaTech is ISO 27001 certified and aligned to ISO/IEC 42001, operates from Accra, Ghana, and serves clients across North America, Europe, Africa, and the Caribbean. Everything AdwumaTech builds belongs to the client who runs it.

Which AI Use Case Should You Build First? The Five-Axis Method AdwumaTech Uses to Rank an Enterprise AI Portfolio