A strong proposal should let you answer a practical question: can these engineers take responsibility for this work inside our product and operating environment?
Resumes, hourly rates, and a list of AI tools give you pieces of the answer. You also need to see how the proposed people investigate a problem, make decisions, verify changes, and communicate when something goes wrong.
This guide walks through that evaluation using one accounting-integration example. It includes a brief you can adapt, questions for a technical session, and a comparison matrix. It also explains how developers can turn AI assistance into useful automation, how to select the first workflows to improve, and why these capabilities matter as the economics of human work changes.
If you are still deciding between an individual engineer, a dedicated squad, and a scoped project, start with our guide to dedicated teams vs staff augmentation. Here, the focus is choosing the people and working arrangement for an ongoing engineering need.
Define the work before you define the team
Start with a workstream you can describe. “We need four senior developers” is a staffing request. “We need to connect our accounting product to an external system while preserving an auditable transaction history” gives a prospective team something to investigate.
Write a short brief covering the outcome, current system, constraints, and people who can answer questions. Use the same brief with every provider so their proposals address the same problem.
For this guide, consider a hypothetical accounting SaaS integration. The external API has rate limits. Events can arrive more than once or out of order. A request may time out after the external system has already processed it. Existing customers must keep working during the rollout.
| Brief field | Example to discuss with the proposed team |
|---|---|
| Desired outcome | Synchronize approved transactions and let an operator trace what happened to each one. |
| Current environment | An existing backend, an external API, a staging environment, and a working release process. |
| Known constraints | Rate limits, duplicate events, ordering, partial failures, and customer data boundaries. |
| Client ownership | A product owner prioritizes work; a technical lead reviews architecture and approves production access. |
| Acceptance evidence | Agreed tests for retries and duplicates, a demonstrated reconciliation path, and an observable rollout. |
| Open questions | Expected volume, API guarantees, acceptable delays, retention needs, and which changes require client approval. |
This is an illustrative exercise, not a description of a SOTR client engagement. Its purpose is to make the evaluation specific enough that different answers become visible.
Do not lock the headcount before discussing the brief. One specialist may cover a contained gap. A wider workstream may need coordinated engineering, QA, and technical leadership. If the outcome or dependencies are still unclear, define a discovery scope before committing to a larger team.
Meet the engineers who would actually join
Ask to meet the proposed people and establish which roles they would hold. A company's portfolio helps you understand its history. It does not show what a particular engineer contributed or how that person will approach your systems.
Choose a relevant past decision and explore it:
- What part did you personally design, implement, or operate?
- Which constraint changed your original approach?
- What failed, and how did you find out?
- What evidence supported the fix?
- Which decision belonged to the client or another team?
Listen for a clear distinction between individual contributions and team results. An engineer who can explain a tradeoff, including what they would do differently, gives you more to evaluate than a polished list of technologies.
Use authorized examples or a new exercise. There is no need to ask candidates to share another client's source code, credentials, or confidential architecture. You can assess reasoning without crossing those boundaries.
Also discuss the working relationship. Who can resolve a product question? Who reviews code? How will the engineer raise a dependency that threatens a milestone? These answers affect whether additional capacity will help your existing team.
Run a technical session around a realistic problem
A focused discussion can reveal more than a broad technology quiz. Share the integration brief in advance and explain what you want to assess. Keep the exercise small enough to discuss without requiring free production work.
One possible 60-minute agenda is:
| Time | Discussion | What to listen for |
|---|---|---|
| 10 minutes | Clarify requirements and constraints | Questions about volume, API behavior, ownership, and unacceptable outcomes. |
| 20 minutes | Explore a design and alternatives | An explanation of why an approach fits the stated constraints. |
| 15 minutes | Walk through failures and tests | Duplicate events, uncertain timeouts, recovery, and useful operational signals. |
| 10 minutes | Propose a first small change | A reviewable scope with clear acceptance evidence. |
| 5 minutes | Summarize open risks | An honest account of what still needs investigation. |
For the accounting example, ask: “The API times out after accepting a transaction. What happens when we retry?”
A weak answer jumps directly to a library and a completion date. A more useful answer investigates whether the API supports idempotency keys, how transaction identity is recorded, and how to reconcile an uncertain result. The engineer should explain which behavior they would test and what they would need to confirm in the API documentation.
There can be several defensible designs. Evaluate how the engineer connects assumptions, decisions, and evidence. Avoid turning your preferred implementation into an unstated answer key.
Ask about AI during the same session. Where would it save effort? Which suggestion would require independent verification? What would the engineer do if generated code passed a happy-path test but mishandled a retry? That connects tool use to the actual work you are buying.
Evaluate how the team uses AI to improve developer productivity
AI can reduce the effort needed to get from a question to a first implementation. Developers can use it to explore unfamiliar code, draft repetitive changes, propose tests, investigate errors, and prepare documentation. A team can use that time to examine more alternatives, cover more failure cases, or spend more attention on architecture and product behavior.
The role of the engineer expands around those faster drafts: selecting the right problem, supplying relevant context, evaluating suggestions, and taking responsibility for the accepted result. A prospective provider should be able to show how those activities fit together.
What the evidence supports
A study published in Management Science in February 2026 pooled three randomized field experiments involving 4,867 developers. It estimated a 26.08% increase in completed development tasks among developers using the AI assistant, with noisy results that varied across experiments. The measured task output was pull requests; the result does not establish an equivalent increase in deployed customer value or a guaranteed gain for your project. Read the study.
DORA's analysis of its 2025 research reports an association between greater AI adoption and both higher delivery throughput and higher delivery instability. Its March 2026 discussion also describes engineers using AI for drafting, information seeking, testing, and review, while spending effort verifying the results. These are findings about observed relationships and reported workflows, not proof that AI alone causes either outcome. Read DORA's analysis.
The practical implication for a buyer is to look for a team that can turn assistance into accepted work. A productivity claim should name the task, the baseline, and the quality checks behind the result.
Where AI can help in the integration example
| Development activity | Useful AI assistance to evaluate | What the engineer still needs to establish |
|---|---|---|
| Understanding the system | Find relevant modules and draft a summary of the existing flow. | Check that summary against the actual code and current behavior. |
| Implementation | Draft a payload adapter or a small refactor following repository conventions. | Confirm the API contract, business rules, error handling, and maintainability. |
| Testing | Suggest cases for duplicate events, rate limits, and timeouts. | Define expected results independently and verify meaningful failure coverage. |
| Investigation | Propose explanations for a failed request using approved diagnostic context. | Reproduce the behavior and test the explanation against evidence. |
| Handover | Draft operational notes and a summary of the change. | Check that the notes match what was implemented and how recovery actually works. |
Suppose AI produces an adapter that retries every timeout. An engineer can reject that suggestion because the external system may already have accepted the transaction. They can then use AI to help draft tests around a chosen idempotency strategy, while validating that strategy against the external API's documented guarantees.
That sequence shows how AI can improve delivery: less repetitive drafting, more attention to the behavior that makes the integration reliable. Generating additional code without resolving the uncertain transaction would leave the essential work unfinished.
Ask for a repeatable workflow
Have the provider walk through one change from requirement to review:
- Define acceptance before implementation. Write down expected behavior and important failure cases.
- Use approved context and tools. Establish what code and data may be supplied to AI tools under the client's rules.
- Keep changes small. Make the result understandable enough to review and test.
- Verify independently. Check source documentation, run meaningful tests, and assign a human reviewer accountable for acceptance.
- Learn from delivery. Review defects and rework to identify where assistance saved effort and where it introduced overhead.
DORA's guidance on small batches supports keeping feedback rapid as the volume of changes grows. The relevant question is whether the team can maintain that feedback when drafting becomes faster.
Measure useful output across the whole workflow
Agree on a baseline for comparable work and include the client's review effort. Track time to accepted delivery, review waiting time, rework, escaped defects, and change failures alongside the product outcome. Account for tooling costs and time spent correcting suggestions.
Lines generated, prompts submitted, or pull requests opened can describe activity. They do not establish that a customer received a useful change. For example, a faster adapter draft is valuable if the team can reach the same or better acceptance standard with less total effort. If review and correction consume the saving, the workflow needs adjustment.
During selection, ask the engineers to explain an AI suggestion they rejected and show the checks behind a change they accepted. That is concrete evidence of how they use assistance and exercise judgment.
Turn faster development into useful automation
There are two opportunities to evaluate. AI can help engineers build software more efficiently. The software they build can also remove repetitive work from your business. A faster implementation is useful; a reliable workflow that saves effort every week can create a continuing benefit after that implementation is complete.
In the accounting example, developers might use AI to help write the integration. The integration can then reduce repeated data entry, surface unmatched transactions, and give operators a clear exception queue. These are different improvements, and they need separate acceptance criteria. An impressive coding demonstration does not establish that the business workflow is better.
Start with work that happens repeatedly
Ask the people doing the work to show you a recent instance from start to finish. Record where information arrives, what gets copied, which decisions follow a rule, who waits for whom, and what happens when the usual path fails. Include the checking and cleanup that people often omit when describing their process.
Look for recurring effort with a clear definition of a correct result. For a development team, that might be preparing a test environment, collecting diagnostics, or assembling release evidence. For a business team, it might be reconciling records, routing requests, or preparing a report from consistent inputs.
Prioritize a workflow whose result you can verify quickly and whose failure you can recover from. Frequency, effort, error cost, and the number of people interrupted all matter. The longest task is not automatically the best first candidate.
Choose the right kind of automation
Some tasks need a script or a workflow engine. Others benefit from AI interpreting variable inputs. Combining the two can be useful when a model can prepare a suggestion and a deterministic system can validate what happens next.
| Situation | Approach to evaluate | Example |
|---|---|---|
| Stable inputs and explicit rules | Deterministic automation | Check required fields, reject duplicate event IDs, or assemble a report from known data. |
| Variable text that needs interpretation | AI assistance with review appropriate to the task | Summarize a support request or suggest an exception category. |
| Interpretation followed by a controlled action | A hybrid workflow | Extract candidate fields, validate them against a schema and business rules, then route exceptions to an operator. |
| An unclear process with disputed outcomes | Process clarification first | Resolve who can approve an adjustment before implementing an automated approval path. |
For example, a model may help explain an unfamiliar exception message. It should not be asked to invent the arithmetic that determines whether two transactions balance. Keep explicit rules explicit; use AI where interpreting context is the actual challenge.
Eight practical ways to make automation pay off
-
Remove unnecessary steps first. If a report is never used, producing it automatically preserves waste. Ask what decision the output supports, then simplify the workflow before implementing it. A smaller process usually needs fewer integrations and less maintenance.
-
Define success and exceptions together. “The sync runs” is incomplete. Specify what should happen to a valid event, a duplicate, a missing field, an unavailable dependency, and an uncertain result. Decide which cases can continue automatically and which need a person. This gives developers a usable boundary for implementation and tests.
-
Automate a narrow slice first. Start with one record type, one environment, or one stage of a workflow. In the example, validate an incoming event and prepare a proposed mapping before enabling the external write. Expand after observing representative cases. Each step should answer a question about correctness or saved effort.
-
Make AI output checkable. Ask for structured results when the next stage expects fields. Validate required values and reference data in code. For a summary, preserve links to the underlying records so a reviewer can verify it. A plausible explanation is not sufficient evidence that the underlying data is correct.
-
Design for retries and partial completion. Scheduled jobs, APIs, and agents can all fail halfway through. Record what completed, prevent duplicate effects where required, and provide a way to resume or reconcile. In the accounting example, retrying a timed-out write must account for the possibility that the transaction already exists.
-
Make the workflow observable. An operator should be able to answer what ran, what changed, what failed, and what needs attention. Prefer an actionable exception queue to a stream of undifferentiated notifications. Useful signals let a small team manage a larger volume without inspecting every successful run manually.
-
Give the automation an owner. Name who maintains the integration, reviews rule changes, and handles exceptions. Keep configuration and operating instructions accessible. Include the cost of dependency changes and occasional model reevaluation in the plan; a workflow that nobody understands becomes another support burden.
-
Reinvest the capacity deliberately. Decide what people will do with the time released: respond faster, investigate exceptions more carefully, improve the product, or serve more customers. Include the people who know the process in redesign and training. Saving minutes in isolation will not improve delivery if the next stage remains blocked.
These are implementation choices to discuss with the proposed engineers. Ask them to apply the checklist to your workflow and explain what they would automate first, what they would leave manual, and why.
Work through the economics before scaling
Consider a hypothetical team of four people who each spend 15 minutes per working day assembling the same operational report. Over 20 working days, that is 20 hours of effort per month. If building the automation takes 10 hours and checking and maintaining it takes another two hours in the first month, the first-month net capacity released would be eight hours—assuming the workflow actually removes the original effort. A later month with two hours of upkeep would release 18 hours.
Those figures are illustrative arithmetic, not a benchmark or a SOTR result. Include exception handling, infrastructure, licenses, and changes in review effort when evaluating your own case. Hours released are capacity; cash savings depend on what costs actually change. The business benefit may instead be a shorter response time or the ability to handle more work with the same team.
Compare the maintained automation with the process it replaces over a realistic period. Review both successful runs and exceptions. If the team spends the saved time fixing unreliable outputs, improve the design before expanding it.
Use the first month to learn
A practical sequence is to observe the current workflow and establish a baseline, then implement one narrow slice, run it alongside the existing process on representative cases, and review the results with its users. Move to normal operation when agreed acceptance conditions are met; retain a way to inspect or recover the work.
At the review, ask whether total effort fell, whether exceptions are easier to resolve, and whether the result arrived sooner for the person who needs it. Record what changed in volume and task difficulty so the comparison remains meaningful. The calendar is a planning aid; it should not force a rollout before the evidence supports it.
Why automation matters in a possible post-labour economy
Post-labour economics explores a future in which machines perform a much larger share of economically useful work, and human employment plays a smaller role in production and potentially in distributing income. In this guide, it is a scenario for thinking about business design. It is not a prediction that jobs disappear on a known schedule.
Automating tasks does not, by itself, establish that a society has reached that future. Demand can expand, new tasks can appear, and the gains can be distributed unevenly. The IMF's January 2026 discussion of skills and AI describes new skill demand alongside disruption and emphasizes the role of policy and preparation. That supports planning for changing work; it does not establish an inevitable end to employment.
Why invest before the long-term outcome is clear?
The near-term case for automation can be evaluated today. A useful workflow lets a business repeat a verified operation with less incremental manual effort. It can reduce waiting between teams, make service more consistent, and preserve operating knowledge in a form that others can inspect and maintain. These benefits can matter even if a post-labour economy never emerges.
The strategic question becomes: what capabilities should keep improving as the volume of work grows? If every additional customer requires the same avoidable copying, checking, and coordination, growth consumes more human effort along that path. A maintained integration or reusable delivery pipeline can change that relationship for the tasks it handles.
The gains are not free or unlimited. Systems need maintenance, exceptions need attention, and capacity elsewhere can become the next bottleneck. For that reason, invest in a sequence of measurable improvements rather than assuming that adopting agents will make an entire business autonomous.
What changes in the developers you need?
As more drafting and routine execution become automated, it is useful to hire engineers who can understand a business process, identify the right automation boundary, connect systems, and verify outcomes. Their contribution includes deciding which work should disappear, which rules belong in software, and which judgment must remain with a person.
In the hypothetical accounting integration, that means asking what an operator is trying to accomplish before automating a sequence of clicks. A good design might eliminate the copying step, make duplicate handling automatic, and reserve human attention for genuinely ambiguous exceptions. The value lies in that redesigned workflow and its reliable operation.
During vendor selection, ask: “If our transaction volume doubled, where would people still spend their time?” Then ask which part the proposed team could improve, how it would measure the benefit, and who would maintain the result. This connects the long-term automation discussion to a concrete engineering decision.
Prepare people as well as systems
People need opportunities to learn the new workflow, question its outputs, and develop the expertise to improve it. Pair engineers and operators during the pilot, document why important rules exist, and make exceptions part of the learning process. Removing repetitive work without transferring knowledge can leave the remaining team dependent on a system it cannot assess.
A company's automation plan also does not answer the wider questions of income distribution, access, or social protection raised by post-labour economics. Keep those questions visible while making a narrower, defensible commitment: build useful capabilities, measure who benefits and who absorbs new work, and help people adapt as responsibilities change.
Compare proposals against the same evidence
Keep a simple record of what each provider has demonstrated and what remains open. Avoid reducing everything to a single weighted score that hides a critical unanswered question.
The entries below illustrate how a buyer might complete a comparison for the hypothetical integration. They are not assessments of real vendors.
| Criterion | Evidence to request | Example evaluation note |
|---|---|---|
| Proposed people | Interviews with the engineers and a clear account of their relevant contributions. | Evidence received: backend engineer explained recovery work; proposed QA role still needs an interview. |
| Technical approach | A discussion of the same brief, alternatives, and failure cases. | Clarification needed: reconciliation after an uncertain timeout is unresolved. |
| AI-assisted work | A small change, verification steps, and an explanation of accepted and rejected suggestions. | Evidence received: engineer identified unsafe retry behavior and demonstrated the corresponding test. |
| Business automation | A workflow map, baseline, exception path, and named operating owner. | Clarification needed: establish who handles unmatched transactions and how total effort will be compared. |
| Quality and release | Acceptance criteria, review responsibilities, and a rollout approach. | Clarification needed: identify who authorizes production release. |
| Decision ownership | Named client and provider owners for priorities, architecture, and escalation. | Evidence received: client owns product priorities; technical escalation path is documented. |
| Continuity | A documentation and transition proposal. | Not demonstrated: coverage if the integration engineer becomes unavailable. |
| Commercial scope | Capacity, included roles, exclusions, and support expectations. | Clarification needed: determine whether technical leadership and operational support are included. |
Resolve an important gap directly. A vendor's overall presentation should not compensate for uncertainty about who will build the integration or who will accept the work.
Use references to validate relevant past behavior when an introduction is permitted. Ask about communication, handling of difficult changes, and continuity. A reference from general enterprise engineering should be described accurately; it is not automatically evidence of experience with a particular platform.
Compare the full engagement cost
Two proposals can have similar engineering rates and cover different work. One might include technical leadership and QA coordination. Another may expect your internal team to provide both.
For the same workstream, compare:
- The roles and capacity included in the fee.
- Client time needed for product direction, technical review, and coordination.
- Onboarding, environments, and tooling costs.
- Testing, release preparation, documentation, and operational handover.
- Agreed continuity, support, and transition arrangements.
Include how AI changes those assumptions. Ask where the provider expects savings, whether verification effort is included, and how estimates will be updated using results from your work. A claim that engineers use AI is not enough to justify a shorter delivery commitment.
For automation work, also agree who owns the resulting code, configuration, documentation, and operating knowledge under the engagement terms. Ask what it will take to change providers or maintain the workflow internally. The initial implementation fee is only one part of the cost of a capability your business will depend on.
Separate candidate-matching time from the date the engineer can contribute safely. Interviews, approvals, access, and environment readiness all affect that date. Make the assumptions visible in the proposal rather than treating a single start-date promise as the whole onboarding plan.
Agree on the first delivery evidence before kickoff
Use the first month as a planning horizon, with checkpoints that fit your dependencies and release controls. The following sequence is illustrative; each stage needs an owner and an agreed condition for moving forward.
| Checkpoint | Evidence in the integration example |
|---|---|
| Ready to work | Approved access, a working development environment, relevant API documentation, and named reviewers. |
| First small change | One bounded part of the adapter, reviewed against agreed input and failure behavior. |
| End-to-end validation | A demonstrated transaction path, duplicate-event tests, and an explanation of uncertain-result recovery. |
| Operational handover | Useful signals, a recovery runbook, rollout and rollback decisions, and a prioritized next scope. |
Review progress with the engineers. If an API guarantee is missing, record the dependency and adjust the plan. If AI accelerates a draft, examine whether the next checkpoint arrived sooner without shifting extra work onto reviewers.
As the engagement develops, measure the team or service in context. DORA's software delivery performance metrics provide a reference for throughput and instability. Use trends alongside the product result; avoid turning them into individual engineer quotas or expecting a reliable statistical comparison from a handful of first-month changes.
Know when to pause before hiring a full team
If the first workstream is still undefined, agree on discovery. If nobody can resolve product questions or authorize access, address that readiness gap. If the missing capability is narrow and your existing team can direct it, evaluate one specialist before adding a squad.
AI can make a well-defined task easier to start. It cannot decide your business priorities, grant the necessary access, or settle an ownership question between teams. Clarifying those conditions helps both the engineers and their tools contribute effectively.
SOTR's published enterprise case studies and anonymized accounting SaaS delivery brief describe the scope of its broader engineering work. They are separate from the hypothetical exercise in this guide. Use them as a starting point, then validate the people and responsibilities proposed for your own engagement.
Bring a real workstream to the team discussion
Bring your roadmap, a repetitive workflow worth improving, and the first result you need to accept. Ask to meet the engineers, discuss how they use AI and design automation, and agree what evidence will demonstrate progress.
Explore SOTR's dedicated engineering squads or discuss your engineering scope. The starting point is a team shape and working arrangement that fit the work, with people you can evaluate directly.

