fix: guard RSA refCnt decrement with held lock

[MarkAtwood]

Bug

In wp_rsa_free() (src/wp_rsa_kmgmt.c) the reference-count decrement cnt = --rsa->refCnt; runs unconditionally, outside the if (rc == 0) mutex-lock-success guard. If wc_LockMutex() fails, two threads can race the unlocked decrement, both observe cnt == 0, and double-free the object.

This is an asymmetry with the sibling wp_rsa_up_ref() in the same file, which correctly guards its increment behind a successfully-held lock.

Fix

Move the decrement inside the lock-success branch and default cnt = 1. On the rare lock failure the object is leaked rather than double-freed (leak-safe), and in the normal path the decrement is atomic with the held lock.

Verification

• Compiled the patched translation unit (default multi-threaded build; --enable-singlethreaded defaults to no, so WP_SINGLE_THREADED is undefined): gcc -c src/wp_rsa_kmgmt.c -> exit 0, no errors.
• Preprocessed output confirms the compiled body now performs the decrement only inside if (rc == 0).

Reported by static analysis (Fenrir finding 515). Severity is low: wolfSSL's pthread wc_LockMutex essentially never fails for a valid normal-type mutex, so this is a latent defect.

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

Fixes a potential double-free race in wp_rsa_free() by ensuring the RSA reference count decrement occurs only while the mutex is successfully held, matching the locking pattern used by wp_rsa_up_ref().

Changes:

• Initialize cnt to a non-zero default to avoid freeing on mutex lock failure (leak-safe fallback).
• Move --rsa->refCnt inside the rc == 0 (lock acquired) branch.
• Preserve unlock behavior only for the successful lock path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.