First Nations management research
How to Measure Economic Development Beyond Jobs
Before renewing an economic-development initiative, distinguish the activity it supports, the additional benefit it creates and the people who receive that benefit. A larger job count can conceal a weaker use of scarce resources.
The consequential question is whether to continue, redesign, expand or retire a particular intervention. That requires an economic benefit assessment with explicit uncertainty, not a longer list of indicators.
Test the renewal decision
- What changed because of this initiative, compared with a credible alternative?
- Who gained, who carried costs and who remained outside the opportunity?
- Would the funding decision change under a less favourable explanation?
Start with the commitment leadership can change
An economic-development office might report new businesses, placements, contracts and investment attracted. These are useful records of activity. They do not, by themselves, tell Council whether another year of supplier support, an enterprise subsidy or a training partnership is the best available commitment.
Define the decision first: renew a specific program at its present scale, change its eligibility or delivery, redirect funding to a different constraint, or conclude it. State the available budget, decision date and obligations to participants. Otherwise, evaluation can become an open-ended search for favourable facts after a renewal has effectively been promised.
Arcus’s central analytical proposition is to separate three judgments: the size and durability of the additional benefit; its distribution and acceptability under the Nation’s priorities; and the confidence that the intervention contributed to it. Strong performance on one does not settle the other two. An initiative can produce measurable additional earnings while leaving the intended participants behind. It can serve a valued purpose while the evidence remains too weak to support expansion.
This article concerns funding and program-design choices. For definitions and routine reporting of community priorities, see measuring progress against community priorities. For coordinating enterprise and community mandates, see economic and community development.
What the evidence establishes—and leaves unresolved
Indigenous Services Canada’s 2023 evaluation of Indigenous Entrepreneurship and Business Development drew on 45 interviews, 441 entrepreneur survey responses and discussion circles. Finding 9 identified a mismatch between reporting measures and outcomes entrepreneurs valued, alongside inconsistent impact data. Respondents described success in terms that included independence and supporting family and community. The evaluation supports questioning output-only reporting; its uneven representation and data gaps do not establish a causal return on each program.1
Evidence of weak measurement should also be updated. The Auditor General’s October 2025 follow-up found satisfactory progress on all three recommendations from its 2018 socio-economic gaps audit. That is a finding about implementing audit recommendations, including improvements to education data. It does not demonstrate the additional economic benefit of an individual Nation’s project.2
Treasury Board Secretariat’s theory-based evaluation guidance explains how evidence about mechanisms and external influences can support a contribution claim. It also cautions that such analysis need not quantify the size of an intervention’s effect.3 The practical implication is to distinguish a credible explanation from a measured causal estimate. Neither an attractive story nor an elaborate spreadsheet eliminates uncertainty.
Why job counts can send funding to the wrong intervention
Counting rules can reward short activity. Twenty people hired for a brief project and twelve people working steadily through the year are different results. Report people, paid hours, duration and continuity separately. State whether jobs are new, maintained, seasonal or replacement hires. A full-time-equivalent calculation standardizes hours; it does not describe job quality, worker choice or household security.
Organizational incentives can obscure the real constraint. A team funded to deliver workshops may optimize attendance even when the missing condition is a purchase order, transport or working capital. Before expanding the same activity, test whether participants reached the next stage and where the sequence broke. The appropriate response may be a purchasing-process change or a differently designed support, rather than more training.
Gross gains can include activity that would have occurred anyway. A hiring subsidy might accelerate a planned recruitment or reach people whom an employer otherwise would not hire. Those are different contributions. Ask employers about recruitment plans made before the intervention and compare them with dated vacancies, payroll and participant experience. Treat retrospective statements as evidence with incentives attached.
One local gain can displace another. A supported business may win customers from an existing member-owned firm. That does not automatically make the intervention undesirable: service quality, prices or resilience may improve. It does mean that the supported firm’s growth cannot simply be labelled growth for the whole local economy. Define the geographic and membership boundary before making a net claim.
Support costs may sit outside the program budget. Finance staff, enterprise supervisors, participant travel and facilities can carry commitments that the economic-development office does not record. A renewal comparison needs the relevant additional resources across organizations, including recurring support after launch.
Distinguish three explanations for apparent success
The intervention removed a binding barrier. Evidence should connect the support to a changed capability or opportunity: a supplier completed a required certification, secured a contract it could not previously serve and delivered successfully. Examine the sequence and whether the improvement persists after intensive support ends. A completed course without a changed opportunity is weaker evidence.
The market or another organization drove the change. A new major buyer, rising product prices or an unrelated infrastructure improvement could explain higher sales and hiring. Compare timing, exposed and unexposed activities where meaningful, and whether the claimed mechanism actually operated. Recognize shared contributions rather than attributing the entire result to one budget.
The program selected people already most likely to succeed. Participants may have stronger networks, more capital or fewer barriers than eligible non-participants. Compare their starting circumstances and document who could not access the support. A simple participants-versus-non-participants comparison can be misleading when these differences are ignored.
These explanations can coexist. The evaluation should state which evidence would weaken its preferred explanation. Where credible comparison data are unavailable, use a bounded contribution judgment and test a range of plausible effects. Do not present an assumed percentage as a research finding.
A worked renewal comparison: the ranking changes
Consider two entirely fictional one-year employment-support options. A costs $240,000 in combined program and additional delivery resources and supports 20 positions, each averaging 30 paid hours for 26 weeks. B costs $300,000 and supports 12 positions, each averaging 32 paid hours for 50 weeks. The costs are assumed to include all additional public delivery resources for that year; they are not the employers’ total payroll.
Counting positions alone gives $12,000 per position for A and $25,000 for B. Using a declared comparison convention of 1,950 paid hours per full-time-equivalent year, A supports 8 FTE-years and B about 9.85. This is a comparison unit, not a prescribed employment standard.
Now test illustrative additionality assumptions. If half of A’s supported hours and three quarters of B’s would not otherwise occur, the additional amounts are 4 and approximately 7.38 FTE-years. Cost per additional FTE-year becomes $60,000 for A and $40,625 for B. The apparent ranking reverses. These fractions are assumptions for sensitivity analysis, not measured effects.
If B’s additionality were only 40%, its cost rises to about $76,172 per additional FTE-year, changing the ranking again. Holding A’s assumptions fixed, B needs additionality above roughly 50.8% to have the lower cost on this measure. That threshold identifies a useful research question: is the evidence strong enough to believe B creates at least that share of additional hours?
The arithmetic does not choose the program. A may reach people facing greater barriers, while B may offer better progression or require travel that participants find impractical. Record earnings and employment conditions, access, relevant participant experience and the Nation’s chosen objectives alongside the cost comparison. An inexpensive outcome that misses the approved purpose is not automatically better.
This is a narrow cost-effectiveness illustration, not a social return on investment or a complete cost-benefit analysis. It excludes longer-term effects and assumes no unmeasured displacement. Any real application must examine those assumptions and the appropriate time horizon.
Assess benefits beyond employment without counting money twice
Build separate evidence records for outcomes that matter to the actual intervention. A supplier initiative might be assessed through repeat business won without continuing assistance, increased delivery capability and the distribution of opportunities among eligible firms. A Nation-owned investment might contribute reliable distributable cash, useful infrastructure or greater control over a strategic service. A land-based enterprise may have conditions concerning access and stewardship that financial measures cannot replace.
Keep the financial perspective explicit. Enterprise sales are not the same as earnings retained locally. A payment to a supplier becomes its revenue; its wages and profits are uses of that revenue, so adding all three as separate benefits overstates the total. A distribution from a Nation-owned enterprise and the subsequent spending of that distribution are also different stages of the same funds.
Show the Nation’s fiscal position separately from participant gains and wider economic effects. A program can be affordable to the Nation because a partner pays part of the bill while still consuming substantial resources overall. Conversely, a worthwhile wider benefit may require a recurring local subsidy that the Nation cannot sustain. Both questions belong in the decision.
Use explicit conditions for priorities that leadership determines should not be traded away. If an option breaches an approved land-use condition or excludes the intended beneficiaries, a high numerical score elsewhere should not silently compensate for it. Do not assign invented dollar values to culture, relationships or autonomy merely to produce one aggregate ratio.
Original Arcus analytical synthesis
The economic-benefit renewal test
Use this decision record for one material commitment. It organizes established evaluation concepts for a funding decision; it is not an empirically validated scoring instrument.
Define the choice
Name the intervention, alternative, beneficiaries, period and resources leadership can change. Identify existing obligations before considering withdrawal.
Test the additional benefit
Separate observed activity from supported causal claims. Examine what would otherwise happen, displacement, external influences and persistence.
Locate gains and burdens
Identify who receives opportunity, who bears cost and whether the result meets the Nation’s approved conditions. Preserve non-negotiable requirements.
Reconcile the full commitment
Distinguish fiscal affordability, total delivery resources and participant costs. Exclude double counting and expose recurring obligations.
Find the decision-changing uncertainty
Test credible alternative assumptions. Identify the evidence that could change the ranking, scale or timing of the next commitment.
Make a bounded decision
Renew, redesign, expand cautiously or conclude. State the rationale, transition arrangements, review point and evidence required before further commitment.
Choose an evaluation approach proportional to the decision
A focused contribution review fits a smaller initiative when leadership needs to understand the mechanism and repair a specific break. Reconcile existing records, test the main alternative explanation and obtain authorized participant and delivery perspectives. It can support a redesign without claiming a precise net effect.
A comparative evaluation is more useful for a substantial expansion when suitable comparison data exist. An evaluator should assess baseline differences, timing, contamination and whether comparison groups are credible. More data do not fix a comparison in which the groups differ on the very factors driving success.
A staged renewal with learning conditions fits a plausible intervention with consequential uncertainty and manageable exposure. Fund a defined next stage while collecting evidence on the disputed mechanism. Specify in advance what warrants expansion or redesign. Avoid repeated short renewals that leave participants and staff carrying uncertainty without resolving it.
Redirection or orderly conclusion becomes credible when the mechanism does not operate, the intended group is not reached or a feasible alternative offers stronger benefits at an affordable cost. Account for notice, existing commitments, participant support and retention of useful capabilities. Ending a funding stream need not mean abandoning its underlying objective.
Assign responsibility for evidence and the renewal decision
The authorized leadership body sets the intended benefit and decision conditions. The economic-development lead documents the intervention and alternatives. Finance reconciles costs, funding timing and recurring commitments. Delivery partners provide agreed evidence; the evaluation lead challenges the explanation. Include people affected through a process the Nation approves, without treating favourable testimonials as a representative impact estimate.
Information governance must precede new collection. FNIGC describes ownership, control, access and possession as First Nations principles that respect each Nation’s decisions about its information.4 Establish permitted use, access, custody and reporting arrangements for linked employer and participant records. Disaggregate only where useful and authorized, and protect against identifying people in small groups.
Begin with an evidence inventory before commissioning surveys. Budget for record reconciliation, analysis, participant engagement, community interpretation and a decision meeting. Recurring monitoring needs a named owner and funded time; a one-time evaluation contract does not supply that capacity. Where information is sparse, focus spending on the uncertainty most likely to change the commitment.
At the renewal meeting, present the preferred option alongside its strongest competing explanation and a feasible alternative. Record the decision, the residual uncertainty, the next evidence milestone and who can authorize a change. Success is a more defensible commitment and a clearer response to evidence—not simply a report with more indicators.
Connect evidence to the next commitment
Evaluate the benefits leadership is funding
Arcus can help leadership diagnose weaknesses in the benefit case, evaluate competing explanations and delivery options, decide renewal conditions, design an evidence and cost model, implement a proportionate review process and measure the results that inform the next decision. Deliverables can include a benefit-and-cost register, an additionality assessment, an options memorandum and an implementation and evaluation plan.
Scope should reflect the Nation’s priorities, information authority and available evidence. Arcus’s analytical framework does not imply access to confidential data or guarantee a measurable causal estimate.
Discuss economic-development decision supportEvidence and source notes
- Indigenous Services Canada, Evaluation of Indigenous Entrepreneurship and Business Development, 2023; web publication June 2024. Methods, limitations and Finding 9. Program evaluation, not a causal return estimate.
- Office of the Auditor General, Follow-up on Programs for First Nations, October 21, 2025. Progress on the three 2018 socio-economic gaps recommendations; scope is federal implementation.
- Treasury Board Secretariat, Theory-Based Approaches to Evaluation: Concepts and Practices, 2012. Sections 4, 6 and 7; methodological guidance and attribution limitations.
- First Nations Information Governance Centre, The First Nations Principles of OCAP®, undated guidance reviewed October 2026. OCAP® is a registered trademark of FNIGC.
Research reviewed October 19, 2026. The renewal test, management analysis and numerical scenarios are Arcus’s original analytical synthesis, not original empirical research or reported client results. No universal benefit threshold, validated score or search-performance improvement is asserted.