Snapshot 36627
Normalized text
Scripts and page chrome removed; this is what change detection compares.
Business Continuity and Disaster Recovery plan - Subscriptionflow BC-DR Plan Navigation Introduction Platform Overview Service Availability Incident Response Business Continuity Disaster Recovery Communication Plan Data & Backups PCI DSS Governance Definitions Business Continuity & Disaster Recovery Plan Client-Facing Document — Confidential — Compliance Scope: PCI DSS Version: 1.0 Effective: 2026-06-01 Review Cycle: Quarterly Classification: Confidential 1 Introduction Purpose This document describes how SubscriptionFlow (SF) maintains service continuity and recovers from disruptions to ensure that client operations are protected. It outlines our preparation, response processes, recovery targets, and communication commitments for any event that affects platform availability. Scope This plan covers all components of the SubscriptionFlow production platform, including the multi-tenant web application and API layer, subscription and billing data storage, background job processing, webhook ingestion and delivery, third-party integration services, and supporting infrastructure (DNS, CDN, monitoring). 2 Platform Overview SubscriptionFlow is hosted entirely on Microsoft Azure (primary region: East US 2). Web & Application Layer Azure Virtual Machines (Ubuntu Linux) Handles all tenant HTTP requests and business logic Database Azure MySQL Flexible Server (HA enabled) Stores all tenant and subscription data Cache & Queue Azure Cache for Redis (Premium) Manages background job queues and application caching Load Balancer Azure Application Gateway (WAF v2) Distributes traffic; provides DDoS and WAF protection DNS & CDN Cloudflare Global DNS, CDN, and DDoS protection layer Monitoring Azure Monitor, Grafana, Prometheus, Uptime Kuma End-to-end observability from infrastructure to availability 3 Service Availability Commitment 99.9% Platform Availability Target Measured monthly across all production endpoints. Allowable downtime: ≤ 43.8 min/month. Full Platform Outage — RTO < 30 min RPO: < 1 hour Partial Outage — RTO < 2 hrs RPO: < 4 hours Non-Critical Degradation — RTO < 8 hrs RPO: < 24 hours 4 Incident Classification & Response Severity Definition Examples P1 — Critical Complete platform unavailability or data integrity risk All tenants unable to access the platform; suspected data loss P2 — High Significant feature degradation or single-node failure Background job processing stopped; one web node down; payment webhooks not delivering P3 — Medium Limited impact; workarounds available Ancillary integration degraded; performance degradation without data impact Severity Time to Acknowledge Time to First Update Update Cadence P1 — Critical 15 minutes 30 minutes Every 30 minutes until resolved P2 — High 30 minutes 1 hour Every hour until resolved P3 — Medium 4 business hours 8 business hours Daily update 5 Business Continuity Procedures Component Failure Behaviour Continuity Mechanism Single web node failure Traffic automatically routed to healthy nodes Azure Application Gateway health checks Database primary failure Automatic failover to standby (60–120 seconds) Azure MySQL HA with standby replica Background job workers stopped Jobs remain queued safely; no data lost Redis queue; workers restarted automatically Redis unavailable Async features degraded; sync data operations unaffected Manual restart; Azure Premium tier SLA Cloudflare / DNS issue Direct access to origin remains available Azure Application Gateway PIP as fallback 6 Disaster Recovery Scenarios 1 Database Loss or Corruption ▼ Detect event via monitoring alerts (Azure Monitor / Uptime Kuma). If HA failover has not triggered automatically, initiate via Azure Portal. If data corruption is detected, restore to last known clean point using PITR. Reconnect application nodes to the restored database. Validate data integrity and resume normal operations. Notify clients per communication plan (§7). 2 Web Application Node Failure ▼ Azure Application Gateway detects the unhealthy node and removes it from rotation automatically. Remaining nodes absorb traffic; capacity is monitored. Failed node is restarted or replaced. Once healthy, the node is re-added to the load balancer pool. 3 Background Job Processing Failure ▼ Detect via Grafana queue-depth dashboard or monitoring alert. Restart worker processes. Confirm queue is draining normally. For extended outage, trigger time-sensitive jobs (renewals, invoices) manually. 4 Full Region Failure ▼ Activate incident bridge and notify all stakeholders. Provision replacement infrastructure in an alternate Azure region. Restore database from backup to the new region. Update DNS records via Cloudflare to point to new region. Validate all services and resume operations. ⚠ Cross-region recovery currently has a longer RTO (estimated 4–8 hours). Cross-region geo-redundant backups are on our infrastructure roadmap. 5 Third-Party Integration Failure (GoCardless, Zoho, Stripe, etc.) ▼ SubscriptionFlow platform continues operating normally. Affected webhook deliveries or API calls are retried per provider configuration. Clients are notified if integration downtime will affect their operations. Resolution is coordinated with the relevant third-party provider. 7 Communication Plan Event Notification Method Timeline P1 incident declared Email + Status Page Within 15 minutes of detection P2 incident declared Email + Status Page Within 30 minutes of detection Incident update Email + Status Page Per cadence in §4.2 Incident resolved Email + Status Page Within 30 minutes of resolution Post-incident report Email Within 48 hours of resolution Planned maintenance Email Minimum 48 hours in advance 📧 Incident Communication Template Subject: [SubscriptionFlow] Service Notification — [Date] We are currently [investigating / experiencing] a disruption affecting [scope]. Status: Investigating / Identified / Monitoring / Resolved Impact: [Description of what is affected] Start time: [UTC timestamp] Next update: [UTC timestamp] 8 Data Protection & Backup Summary Data Type Backup Method Frequency Retention Recovery Granularity Tenant subscription data Azure PITR (automated) Continuous 30 days Any second within retention window Tenant subscription data Daily snapshot backup Daily TBD Per-day restore Application configuration Version-controlled (Git) On every change Indefinite Per-commit Monitoring & log data Centralised log aggregation Continuous TBD Per-event 🔒 All production data is encrypted at rest using Azure-managed encryption and in transit using TLS 1.2 or higher. 9 PCI DSS Compliance Area Status Web application firewall (WAF) Active — Azure Application Gateway WAF v2 Data encryption in transit TLS 1.2+ enforced on all client-facing endpoints Data encryption at rest Azure-managed encryption on all data stores Access control Role-based access; least privilege applied Audit logging Centralised log aggregation and monitoring Vulnerability management Regular patching; Azure Advisor reviewed Network segmentation Production isolated in dedicated Azure Virtual Network PCI compliance Fully 10 Plan Governance & Testing Test Frequency Database PITR restore drill Quarterly Application Gateway failover test Quarterly Job worker failure and recovery drill Monthly Full DR tabletop exercise (team walkthrough) Annually Communication plan test Semi-annually 11 Definitions RTO — Recovery Time Objective Maximum acceptable time from incident onset to service restoration RPO — Recovery Point Objective Maximum acceptable data loss measured in time PITR — Point-in-Time Restore Database recovery feature allowing restore to any specific point within a retention window WAF — Web Application Firewall Security layer that inspects and filters web traffic for malicious requests HA — High Availability Architecture where a standby component automatically takes over if the primary fails CDN — Content Delivery Network Distributed network (Cloudflare) that accelerates delivery and absorbs DDoS attacks P1 / P2 / P3 Internal severity classification for incidents — P1 is the most severe