fix: fail cancelled smart transactions through the standard failure path

[cloudonshore]

Explanation

When the transaction relay cancels a smart transaction (e.g. FAILED_GAS_TOO_LOW — the smart tx never lands on chain), SmartTransactionsController marks the associated regular transaction as failed via the TransactionController:updateTransaction action.

updateTransaction only patches state — it does not emit any transaction lifecycle events. As a result, consumers that react to transactionFailed / transactionStatusUpdated are never notified:

• BridgeStatusController marks a bridge item failed only from the transactionStatusUpdated / transactionFailed event. Since that event never fires for a cancelled smart tx, the bridge history item stays PENDING forever — the transaction shows as failed, but the bridge UI remains stuck on "Pending".
• Transaction metrics (Transaction Finalized) are likely under-reported for the same reason.

This was observed in production with bridge smart transactions repeatedly stuck as pending (source tx status: failed, bridge status.status: PENDING), while the relay's batchStatus reported minedTx: cancelled / cancellationReason: too_cheap.

Fix

Route the cancellation failure through the standard fail path so the lifecycle events fire.

• transaction-controller: add a public failTransaction(transactionId, error) method and corresponding TransactionController:failTransaction messenger action. It fails the transaction through the internal #failTransaction, emitting transactionFailed, transactionStatusUpdated, and transactionFinished.
• smart-transactions-controller: markRegularTransactionsAsFailed now calls failTransaction instead of updateTransaction.

References

• Fixes the "smart transaction bridge stuck as pending" reports.

Changelog

@metamask/transaction-controller

• ADDED: failTransaction method and TransactionController:failTransaction messenger action.

@metamask/smart-transactions-controller

• FIXED: Fail the associated regular transaction via TransactionController:failTransaction (was updateTransaction) so transaction lifecycle events are emitted and downstream consumers no longer leave cancelled smart transactions stuck as pending.
  • Consumers must grant the smart-transactions-controller messenger access to TransactionController:failTransaction (previously TransactionController:updateTransaction).

Checklist

• I've updated the test suite for new or updated code as appropriate
• I've updated documentation (JSDoc) for new or updated code as appropriate
• I've highlighted breaking changes using the "BREAKING" category above as appropriate

---

Note

Medium Risk
Touches core transaction failure signaling and requires a messenger permission change for smart-transactions-controller; behavior is narrowly scoped to cancelled smart-tx paths but downstream bridge/metrics depend on it.

Overview
When a smart transaction is cancelled, the linked regular transaction must be marked failed in a way that emits transaction lifecycle events, not only a state patch.

transaction-controller adds public failTransaction(transactionId, error) and TransactionController:failTransaction, routing through the existing internal failure path so transactionFailed, transactionStatusUpdated, and transactionFinished fire. updateTransaction is unchanged and still does not emit those events.

smart-transactions-controller updates markRegularTransactionsAsFailed and #createOrUpdateSmartTransaction to call failTransaction with a SmartTransactionFailed error instead of updateTransaction with a full failed TransactionMeta. Messenger allowed actions swap TransactionController:updateTransaction for TransactionController:failTransaction.

Integrators must grant the smart transactions controller access to TransactionController:failTransaction (replacing updateTransaction for this flow). This fixes bridge and similar flows where the source tx looked failed but bridge history stayed PENDING because subscribers never saw lifecycle events.

^{Reviewed by Cursor Bugbot for commit c72d709. Bugbot is set up for automated code reviews on this repo. Configure here.}