
How often should a crypto exchange run a vulnerability assessment?
NYDFS 23 NYCRR 500.5 sets no fixed calendar cadence for vulnerability scans — it makes your own risk assessment the clock. What that means in practice, and what happens when the risk assessment itself is wrong.
The short answer
New York's cybersecurity regulation doesn't give a crypto exchange a number to put on the calendar for vulnerability scanning. 23 NYCRR 500.5(a)(2) requires automated scans plus manual review of what they miss "at a frequency determined by the risk assessment, and promptly after any material system changes" — no quarterly, no monthly, no fixed floor. Penetration testing is the one item in the same section that does get a hard number: at least annually, per 500.5(a)(1), independent of what the risk assessment concludes. That's a structurally different cadence problem than a casino or card processor faces under a flat quarterly rule, and getting the risk assessment wrong carries its own penalty — NYDFS fined bitFlyer USA $1.2 million in 2023 for exactly that gap.
Who this decision is for
This is written for the CISO, Head of Security, or compliance lead at a crypto exchange holding — or applying for — a New York virtual currency license (the BitLicense, 23 NYCRR Part 200), who has to set a defensible Vulnerability Assessment cadence and justify it to a board or an examiner rather than point to a rule with a number already in it.
It's not for you if your exchange has no New York nexus and instead sits under DORA, MiCA, or another regime — see the note on DORA below, and NIS2 and DORA compliance without the consultant theatre for that annual-testing requirement directly. It's also not for you if you're still deciding between a VA and a full penetration test — that breadth-vs-depth decision is covered in vulnerability assessment vs penetration testing for iGaming, and scoping a penetration test for a crypto exchange covers what a crypto-exchange pentest scope actually names.
Why a BitLicense makes this NYDFS's rule to begin with
A virtual currency license issued under 23 NYCRR Part 200 doesn't sit in a separate compliance track from New York's cybersecurity regulation — it's the trigger for it. 23 NYCRR 500.01(c) defines "Covered Entity" as any person operating under a license, registration, charter, certificate, permit, or similar authorization under the Banking Law, the Insurance Law, or the Financial Services Law, and the BitLicense is issued under the Financial Services Law. NYDFS's own consent order against bitFlyer USA states the link without ambiguity: "by virtue of its license, granted pursuant to the Virtual Currency Regulation, bitFlyer USA is a 'Covered Entity,' pursuant to 23 NYCRR § 500.01(c)." Any exchange operating in New York under a BitLicense is answering to the same cybersecurity regulation as a bank or insurer chartered in the state — the vulnerability-management clock included.
What 500.5 actually requires, split in two
The section covers both engagement types, and it doesn't treat them the same way:
| Requirement | Cadence | What sets it |
|---|---|---|
| Penetration testing — 500.5(a)(1) | At least annually | Fixed floor, written into the rule |
| Vulnerability scans / assessment — 500.5(a)(2) | "At a frequency determined by the risk assessment, and promptly after any material system changes" | Your own risk assessment, not a calendar number |
The annual penetration test is checkable the same way PCI DSS's quarterly external scan is — count the months since the last one. The vulnerability-assessment cadence isn't checkable that way at all. An examiner reviewing your program isn't asking "was it quarterly" — they're asking "does your risk assessment actually justify the frequency you picked, and can you show your work."
The bitFlyer USA case: what a weak risk assessment costs
NYDFS examined bitFlyer USA's cybersecurity program covering November 27, 2017 through September 30, 2020, and concluded the company's risk assessment didn't sufficiently inform the design of its cybersecurity program — a violation of 23 NYCRR 500.09(a), not 500.5 directly. The company settled in a May 2023 consent order for a $1,200,000 civil monetary penalty. The reasoning in that order is the part that matters for VA cadence specifically: it states that a comprehensive risk assessment is "a necessary prerequisite for compliance with the Department's requirements for penetration testing (23 NYCRR § 500.05), audit trails (23 NYCRR § 500.06), review of access privileges (23 NYCRR § 500.07)" and several other sections. In a regime where the risk assessment itself sets your scan frequency, a deficient risk assessment doesn't just risk missing something on the estate — it's a standalone violation an examiner can act on, whatever cadence it happened to produce.
Building a risk assessment an examiner can act on, not around
A cadence that's genuinely risk-based — and defensible as such — has to be able to show its reasoning, not just its output:
- Document the inputs, not just the conclusion. What assets were scored, what threat model drove the scoring, and why the resulting cadence follows from it — not just "we scan monthly."
- Tie cadence to asset criticality, not a blanket schedule. Systems that can initiate or authorize withdrawals justify a tighter cycle than a marketing site; a risk assessment that assigns the same frequency across the whole estate isn't doing the differentiation the rule assumes.
- Re-run the assessment on the trigger events the rule already names. "Promptly after any material system change" isn't optional language — a new custody integration, a new trading-API partner, or a new hosting region are each a reason to reassess before the next scheduled cycle, not at it.
- Keep the annual penetration test on its own fixed clock regardless. 500.5(a)(1) doesn't bend to the risk assessment the way (a)(2) does — treat it as a hard deadline, not a target.
- Expect the assessment itself to be examined, not just the scan reports it produces. bitFlyer's violation was in the assessment's design, not in a missed scan.
What you get each cycle
A Vulnerability Assessment engagement runs 3–5 business days per cycle, with a recurring option scheduled to match whatever cadence your risk assessment sets rather than a fixed quarterly default — a prioritized, de-duplicated findings list ranked by real exposure, concrete remediation guidance per finding, and an audit-ready summary formatted for an examiner, a board, or an internal audit trail. Every automated hit is manually verified before it reaches the report.
Limitations
- This is a cadence framework, not a risk-assessment template. It doesn't replace legal or compliance review of your specific risk-assessment methodology or documentation — confirm your approach directly with counsel or your compliance function.
- NYDFS applies to New York-nexus entities. An exchange with no BitLicense and no New York operations isn't a Covered Entity under this regulation; check which regime — DORA, MiCA, a different state or national framework — actually applies to your license footprint.
- This doesn't cover VA scope. What gets scanned and where its evidence runs out for a crypto exchange specifically isn't addressed here — see scoping a penetration test for a crypto exchange for the adjacent scoping questions on the pentest side.
- A risk-based cadence only works if the risk assessment is maintained. A one-time assessment that never gets revisited after a material change is functionally the same gap NYDFS charged bitFlyer USA for.
Decision framework and next step
If your exchange holds or is applying for a BitLicense, don't default to a quarterly number — build a risk assessment that actually justifies the vulnerability-scan cadence you run, keep the annual penetration test on its own fixed clock regardless, and re-run the risk assessment on every material system change rather than waiting for the next scheduled cycle. If you're not sure your current risk assessment would hold up to the kind of scrutiny bitFlyer USA's didn't, tell us your estate and current schedule and we'll help you set — or defend — the right cadence.
Sources and review
23 NYCRR 500.5 (penetration testing and vulnerability management cadence) and 500.01(c) (Covered Entity definition) are cited to the New York Department of Financial Services' own published regulation text. The bitFlyer USA case — the Covered Entity determination, the 500.09(a) risk-assessment violation, the $1,200,000 penalty, and the "necessary prerequisite" language tying the risk assessment to penetration-testing compliance — is cited to the NYDFS consent order dated May 1, 2023, published on the Department's own site. Author: AnySec Engineering. Published 2026-09-03.
Related reading
- How often should an online casino run a vulnerability assessment? — the fixed-cadence version of this same decision, for operators under PCI DSS or a gaming regulator's quarterly rule.
- Scoping a penetration test for a crypto exchange — what the annual penetration test itself needs to name in scope for an exchange.
- Vulnerability assessment vs penetration testing for iGaming — the breadth-vs-depth engagement decision, before cadence is the question.
- NIS2 and DORA compliance without the consultant theatre — the annual-testing requirement for exchanges under DORA instead of NYDFS.
Keep reading
All insights →Rather not learn this in production.
Talk to the engineers behind these write-ups — no sales script, a straight read on where you stand.
Get a fixed quote
