Snapshot 251474

Document
Registry listing
URL
https://status.ubicloud.com/history.rss
Fetched
HTTP status
200
Content type
text/xml
Fetch mode
api
Size
115534 bytes
SHA-256 (raw)
ed4d7345aefd11940e726a1200ab498a7003abdaaa4bfb75da67459688fdf42e
SHA-256 (normalized text)
651ac2be9d0021fbf6e56594eac91c136f1153cf5f71cd5792311679ce72ef44

Raw (normalized text)

Scripts and page chrome removed; this is what change detection compares.

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<title>Ubicloud Status - Incident history</title>
<link>https://status.ubicloud.com</link>
<description>Ubicloud</description>
<pubDate>Tue, 10 Nov 2026 16:10:52 +0000</pubDate>
<item>
<title>Incident with Git Operations and Pull Requests</title>
<description>
Type: Incident
Duration: 4 days, 8 hours and 11 minutes
Affected Components: GitHub → Actions
Nov 10, 17:55:03 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 10, 17:46:33 GMT+0 - Investigating - We continue to investigate elevated response times for Git operations and pull requests. Actions workflows are recovering. Pull request processing has shown some improvement, but users continue to experience delays and failures. We’re monitoring the impact and working to restore normal service. Nov 10, 17:28:17 GMT+0 - Investigating - Actions is operating normally. Nov 10, 17:12:54 GMT+0 - Investigating - Pull Requests is experiencing degraded performance. We are continuing to investigate. Nov 10, 17:07:48 GMT+0 - Investigating - We continue to investigate elevated response times for Git operations and pull requests. Some pull-request processing and Actions workflows are improving, but delays and failures persist. We&#039;re monitoring the impact and working to restore normal service. Nov 10, 16:39:58 GMT+0 - Investigating - Actions is experiencing degraded availability. We are continuing to investigate. Nov 10, 16:31:01 GMT+0 - Investigating - We&#039;re investigating elevated response times for Git operations and pull requests. GitHub Actions workflows that are triggered from Pull Requests are delayed. We&#039;re working to mitigate the impact and will share updates as we learn more. Nov 10, 16:26:37 GMT+0 - Investigating - Actions and Pull Requests are experiencing degraded availability. We are continuing to investigate. Nov 10, 16:10:52 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 4 days, 8 hours and 11 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>17:55:03</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>17:46:33</var> GMT+0</small><br><strong>Investigating</strong> -
We continue to investigate elevated response times for Git operations and pull requests. Actions workflows are recovering. Pull request processing has shown some improvement, but users continue to experience delays and failures. We’re monitoring the impact and working to restore normal service..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>17:28:17</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is operating normally..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>17:12:54</var> GMT+0</small><br><strong>Investigating</strong> -
Pull Requests is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>17:07:48</var> GMT+0</small><br><strong>Investigating</strong> -
We continue to investigate elevated response times for Git operations and pull requests. Some pull-request processing and Actions workflows are improving, but delays and failures persist. We&#039;re monitoring the impact and working to restore normal service..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>16:39:58</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>16:31:01</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re investigating elevated response times for Git operations and pull requests. GitHub Actions workflows that are triggered from Pull Requests are delayed. We&#039;re working to mitigate the impact and will share updates as we learn more..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>16:26:37</var> GMT+0</small><br><strong>Investigating</strong> -
Actions and Pull Requests are experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 10</var>, <var data-var='time'>16:10:52</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Tue, 10 Nov 2026 16:10:52 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmwaw44rs27qd0rrhlxsiazma</link>
<guid>https://status.ubicloud.com/incident/cmwaw44rs27qd0rrhlxsiazma</guid>
</item>
<item>
<title>Incident with Actions and Checks API</title>
<description>
Type: Incident
Duration: 1 day, 22 hours and 55 minutes
Affected Components: GitHub → Actions, GitHub → Webhooks
Nov 9, 18:59:32 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 9, 18:43:24 GMT+0 - Monitoring - The degradation affecting Actions, Pages and Webhooks has been mitigated. We are monitoring to ensure stability. Nov 9, 18:30:17 GMT+0 - Investigating - Pages is experiencing degraded performance. We are continuing to investigate. Nov 9, 18:24:13 GMT+0 - Investigating - We&#039;re investigating elevated errors affecting GitHub Actions and the Checks API. We&#039;ll share another update as we learn more. Nov 9, 18:23:13 GMT+0 - Investigating - Webhooks is experiencing degraded performance. We are continuing to investigate. Nov 9, 18:14:58 GMT+0 - Investigating - Actions is experiencing degraded availability. We are continuing to investigate. Nov 9, 18:12:37 GMT+0 - Investigating - We are investigating reports of degraded availability for Actions
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 22 hours and 55 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:59:32</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:43:24</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions, Pages and Webhooks has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:30:17</var> GMT+0</small><br><strong>Investigating</strong> -
Pages is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:24:13</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re investigating elevated errors affecting GitHub Actions and the Checks API. We&#039;ll share another update as we learn more..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:23:13</var> GMT+0</small><br><strong>Investigating</strong> -
Webhooks is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:14:58</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>18:12:37</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded availability for Actions.</p>
]]>
</content:encoded>
<pubDate>Mon, 9 Nov 2026 18:12:37 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw9kylzp022o0rrhp7nvgbcm</link>
<guid>https://status.ubicloud.com/incident/cmw9kylzp022o0rrhp7nvgbcm</guid>
</item>
<item>
<title>Incident with Webhooks and Actions</title>
<description>
Type: Incident
Duration: 4 days and 14 hours
Affected Components: GitHub → Webhooks, GitHub → Actions
Nov 9, 11:50:35 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 9, 11:45:07 GMT+0 - Monitoring - The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability. Nov 9, 11:42:04 GMT+0 - Investigating - Webhooks is experiencing degraded performance. We are continuing to investigate. Nov 9, 11:17:28 GMT+0 - Investigating - Webhooks is experiencing degraded availability. We are continuing to investigate. Nov 9, 11:06:35 GMT+0 - Investigating - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Nov 9, 10:47:25 GMT+0 - Investigating - We&#039;ve applied a mitigation and are monitoring for recovery. Nov 9, 10:28:19 GMT+0 - Investigating - We&#039;re investigating delays in Actions runs start times and webhook deliveries. Nov 9, 10:00:35 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Webhooks
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 4 days and 14 hours</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>11:50:35</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>11:45:07</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>11:42:04</var> GMT+0</small><br><strong>Investigating</strong> -
Webhooks is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>11:17:28</var> GMT+0</small><br><strong>Investigating</strong> -
Webhooks is experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>11:06:35</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>10:47:25</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;ve applied a mitigation and are monitoring for recovery..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>10:28:19</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re investigating delays in Actions runs start times and webhook deliveries..</p>
<p><small>Nov <var data-var='date'> 9</var>, <var data-var='time'>10:00:35</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Webhooks.</p>
]]>
</content:encoded>
<pubDate>Mon, 9 Nov 2026 10:00:35 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw93ghbt1ppa0kld2xgfkdhl</link>
<guid>https://status.ubicloud.com/incident/cmw93ghbt1ppa0kld2xgfkdhl</guid>
</item>
<item>
<title>Increase in timeouts for some unauthenticated users</title>
<description>
Type: Incident
Duration: 4 days, 20 hours and 30 minutes
Affected Components: GitHub → API Requests
Nov 8, 12:32:35 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 8, 12:29:30 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Nov 8, 11:56:20 GMT+0 - Investigating - The degradation affecting API Requests has been mitigated. We are monitoring to ensure stability. Nov 8, 11:50:49 GMT+0 - Investigating - We&#039;ve identified a problematic component and are working to redeploy it. Nov 8, 11:20:57 GMT+0 - Investigating - API Requests is experiencing degraded performance. We are continuing to investigate. Nov 8, 11:16:19 GMT+0 - Investigating - We are continuing to investigate degraded performance primarily for unauthenticated users.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 4 days, 20 hours and 30 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>12:32:35</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>12:29:30</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>11:56:20</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting API Requests has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>11:50:49</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;ve identified a problematic component and are working to redeploy it..</p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>11:20:57</var> GMT+0</small><br><strong>Investigating</strong> -
API Requests is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>11:16:19</var> GMT+0</small><br><strong>Investigating</strong> -
We are continuing to investigate degraded performance primarily for unauthenticated users..</p>
]]>
</content:encoded>
<pubDate>Sun, 8 Nov 2026 10:36:05 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw7qvhkn6nac0kp3wqv0b7do</link>
<guid>https://status.ubicloud.com/incident/cmw7qvhkn6nac0kp3wqv0b7do</guid>
</item>
<item>
<title>Degraded performance for multiple GitHub services</title>
<description>
Type: Incident
Duration: 1 day, 20 hours and 43 minutes
Affected Components: GitHub → Actions
Nov 6, 00:20:49 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 6, 00:19:59 GMT+0 - Monitoring - This incident has been resolved. All affected services, including Git operations, Issues, Actions, and API access, have recovered and remained stable. Thank you for your patience. We&#039;ll share more details in our availability report. Nov 6, 00:19:21 GMT+0 - Investigating - Git Operations is operating normally. Nov 6, 00:06:28 GMT+0 - Monitoring - We&#039;ve applied a mitigation and are seeing recovery across affected services, including Git operations, Issues, Actions, and API access. We&#039;re monitoring to confirm stability and will post another update by 00:35 UTC. Nov 6, 00:06:11 GMT+0 - Monitoring - The degradation affecting Actions, Issues and Pages has been mitigated. We are monitoring to ensure stability. Nov 5, 23:58:28 GMT+0 - Investigating - Issues is experiencing degraded performance. We are continuing to investigate. Nov 5, 23:36:51 GMT+0 - Investigating - We&#039;re investigating degraded performance and errors affecting several GitHub services, including Git operations, Issues, Actions, and API access. Some users may see failed requests or slow responses. We&#039;ll post another update by 00:05 UTC. Nov 5, 23:36:06 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Pages
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 20 hours and 43 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>00:20:49</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>00:19:59</var> GMT+0</small><br><strong>Monitoring</strong> -
This incident has been resolved. All affected services, including Git operations, Issues, Actions, and API access, have recovered and remained stable. Thank you for your patience. We&#039;ll share more details in our availability report..</p>
<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>00:19:21</var> GMT+0</small><br><strong>Investigating</strong> -
Git Operations is operating normally..</p>
<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>00:06:28</var> GMT+0</small><br><strong>Monitoring</strong> -
We&#039;ve applied a mitigation and are seeing recovery across affected services, including Git operations, Issues, Actions, and API access. We&#039;re monitoring to confirm stability and will post another update by 00:35 UTC..</p>
<p><small>Nov <var data-var='date'> 6</var>, <var data-var='time'>00:06:11</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions, Issues and Pages has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>23:58:28</var> GMT+0</small><br><strong>Investigating</strong> -
Issues is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>23:36:51</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re investigating degraded performance and errors affecting several GitHub services, including Git operations, Issues, Actions, and API access. Some users may see failed requests or slow responses. We&#039;ll post another update by 00:05 UTC..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>23:36:06</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Pages.</p>
]]>
</content:encoded>
<pubDate>Thu, 5 Nov 2026 23:36:06 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw46t5lp0io70rp3zv38xoy3</link>
<guid>https://status.ubicloud.com/incident/cmw46t5lp0io70rp3zv38xoy3</guid>
</item>
<item>
<title>Incident with Actions</title>
<description>
Type: Incident
Duration: 2 days, 4 hours and 24 minutes
Affected Components: GitHub → Actions
Nov 5, 17:42:35 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 5, 17:04:46 GMT+0 - Monitoring - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Nov 5, 16:52:52 GMT+0 - Investigating - Some GitHub Actions workflow runs may experience delays or errors when starting. We’re investigating and preparing a mitigation. Nov 5, 16:50:11 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 2 days, 4 hours and 24 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>17:42:35</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>17:04:46</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>16:52:52</var> GMT+0</small><br><strong>Investigating</strong> -
Some GitHub Actions workflow runs may experience delays or errors when starting. We’re investigating and preparing a mitigation..</p>
<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>16:50:11</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions.</p>
]]>
</content:encoded>
<pubDate>Thu, 5 Nov 2026 16:50:11 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw3scbcx00lm0ro0ky0ge8p9</link>
<guid>https://status.ubicloud.com/incident/cmw3scbcx00lm0ro0ky0ge8p9</guid>
</item>
<item>
<title>Incident with Actions</title>
<description>
Type: Incident
Duration: 1 day, 11 hours and 34 minutes
Affected Components: GitHub → Actions
Nov 4, 18:16:08 GMT+0 - Resolved - Please refer to the combined summary in this related incident: https://www.githubstatus.com/incidents/xr3myzj7ms51 Nov 4, 18:07:51 GMT+0 - Monitoring - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Nov 4, 17:53:11 GMT+0 - Investigating - We have identified an issue with events that might trigger Actions. We have applied mitigations and are monitoring for recovery at this point. Customers may observe delayed status updates for their jobs. Nov 4, 17:40:33 GMT+0 - Investigating - We are investigating reports of degraded availability for Actions
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 11 hours and 34 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>18:16:08</var> GMT+0</small><br><strong>Resolved</strong> -
Please refer to the combined summary in this related incident: https://www.githubstatus.com/incidents/xr3myzj7ms51.</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>18:07:51</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>17:53:11</var> GMT+0</small><br><strong>Investigating</strong> -
We have identified an issue with events that might trigger Actions. We have applied mitigations and are monitoring for recovery at this point. Customers may observe delayed status updates for their jobs..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>17:40:33</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded availability for Actions.</p>
]]>
</content:encoded>
<pubDate>Wed, 4 Nov 2026 17:40:33 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw2eormz05fw0rqevfx6wko3</link>
<guid>https://status.ubicloud.com/incident/cmw2eormz05fw0rqevfx6wko3</guid>
</item>
<item>
<title>Incident with Actions and Pages</title>
<description>
Type: Incident
Duration: 2 days, 7 hours and 39 minutes
Affected Components: GitHub → Actions, GitHub → Webhooks
Nov 4, 17:01:11 GMT+0 - Resolved - On November 4, 2026, between 15:49 UTC and 16:48 UTC, and again between 17:20 UTC and 17:55 UTC, GitHub Actions, Pages, Webhooks, Code Scanning, and Copilot coding agent experienced delays. During this time, Actions workflow runs and job status updates were delayed by up to 25 minutes. Pages deployments dropped to about 4% of normal volume, webhook deliveries dropped to about 58% of normal volume, and Code Scanning analyses and Copilot coding agent sessions were delayed. Delayed events were processed after mitigation. &lt;br /&gt;&lt;br /&gt;This incident was caused by a configuration change to an internal event-routing service that unexpectedly paused delivery of several event streams. The change depended on feature flags that were not set up as intended, which the service did not handle gracefully. We mitigated the first period by rolling back the configuration change at 16:14 UTC. The code change itself was not reverted by the rollback, so a subsequent deployment reintroduced it. We mitigated the second period at 17:40 UTC by rolling back again, then processed queued events and cleared backlogs. &lt;br /&gt;&lt;br /&gt;We are taking several steps to prevent a repeat. The routing service will keep its existing behavior when a feature toggle is missing, so a misconfigured flag can no longer pause delivery. We are adding a pre-deploy check that each routing flag exists and is registered under the correct owning service, and we are making flag ownership less error-prone to set. We are also improving alerting so we detect paused event delivery sooner. Nov 4, 16:49:27 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Nov 4, 16:48:03 GMT+0 - Monitoring - The degradation affecting Actions and Pages has been mitigated. We are monitoring to ensure stability. Nov 4, 16:45:48 GMT+0 - Investigating - The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability. Nov 4, 16:34:45 GMT+0 - Investigating - We have identified an issue with events that might trigger Actions. We have applied mitigations and are monitoring for recovery at this point. Customers may observe delayed status updates for their jobs. Nov 4, 16:17:29 GMT+0 - Investigating - Actions is experiencing degraded availability. We are continuing to investigate. Nov 4, 16:12:55 GMT+0 - Investigating - Webhooks is experiencing degraded performance. We are continuing to investigate. Nov 4, 16:11:38 GMT+0 - Investigating - We are investigating reports of issues with Actions and Pages. We will continue to keep users updated on progress towards mitigation. Nov 4, 16:05:31 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Pages
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 2 days, 7 hours and 39 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>17:01:11</var> GMT+0</small><br><strong>Resolved</strong> -
On November 4, 2026, between 15:49 UTC and 16:48 UTC, and again between 17:20 UTC and 17:55 UTC, GitHub Actions, Pages, Webhooks, Code Scanning, and Copilot coding agent experienced delays. During this time, Actions workflow runs and job status updates were delayed by up to 25 minutes. Pages deployments dropped to about 4% of normal volume, webhook deliveries dropped to about 58% of normal volume, and Code Scanning analyses and Copilot coding agent sessions were delayed. Delayed events were processed after mitigation. &lt;br /&gt;&lt;br /&gt;This incident was caused by a configuration change to an internal event-routing service that unexpectedly paused delivery of several event streams. The change depended on feature flags that were not set up as intended, which the service did not handle gracefully. We mitigated the first period by rolling back the configuration change at 16:14 UTC. The code change itself was not reverted by the rollback, so a subsequent deployment reintroduced it. We mitigated the second period at 17:40 UTC by rolling back again, then processed queued events and cleared backlogs. &lt;br /&gt;&lt;br /&gt;We are taking several steps to prevent a repeat. The routing service will keep its existing behavior when a feature toggle is missing, so a misconfigured flag can no longer pause delivery. We are adding a pre-deploy check that each routing flag exists and is registered under the correct owning service, and we are making flag ownership less error-prone to set. We are also improving alerting so we detect paused event delivery sooner..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:49:27</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:48:03</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions and Pages has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:45:48</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting Webhooks has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:34:45</var> GMT+0</small><br><strong>Investigating</strong> -
We have identified an issue with events that might trigger Actions. We have applied mitigations and are monitoring for recovery at this point. Customers may observe delayed status updates for their jobs..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:17:29</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:12:55</var> GMT+0</small><br><strong>Investigating</strong> -
Webhooks is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:11:38</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of issues with Actions and Pages. We will continue to keep users updated on progress towards mitigation..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>16:05:31</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Pages.</p>
]]>
</content:encoded>
<pubDate>Wed, 4 Nov 2026 16:05:31 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw2bap2g01fa0rqedvttcnqp</link>
<guid>https://status.ubicloud.com/incident/cmw2bap2g01fa0rqedvttcnqp</guid>
</item>
<item>
<title>Incident with Pull Requests, Git Operations and Issues</title>
<description>
Type: Incident
Duration: 5 days and 10 minutes
Affected Components: GitHub → Actions, GitHub → API Requests
Nov 4, 11:50:06 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 4, 11:43:52 GMT+0 - Monitoring - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Nov 4, 11:30:42 GMT+0 - Investigating - The degradation affecting API Requests, Git Operations, Issues and Pull Requests has been mitigated. We are monitoring to ensure stability. Nov 4, 11:17:52 GMT+0 - Investigating - We&#039;re currently working on mitigating elevated errors and intermittent failures affecting GitHub, including Pull Requests, Git Operations, and Issues. Some requests may fail or be delayed while we work to restore service. We&#039;ll provide another update as more information becomes available. Nov 4, 11:11:30 GMT+0 - Investigating - API Requests is experiencing degraded performance. We are continuing to investigate. Nov 4, 10:38:49 GMT+0 - Investigating - We&#039;re investigating elevated errors and intermittent failures affecting GitHub including Pull Requests, Git Operations, and Issues. We&#039;ll continue to keep users updated as we work toward mitigation. Nov 4, 09:53:38 GMT+0 - Investigating - Pull Requests is experiencing degraded availability. We are continuing to investigate. Nov 4, 09:50:47 GMT+0 - Investigating - Actions is experiencing degraded performance. We are continuing to investigate. Nov 4, 09:49:55 GMT+0 - Investigating - We are investigating reports of degraded performance for Git Operations, Issues and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 5 days and 10 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:50:06</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:43:52</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:30:42</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting API Requests, Git Operations, Issues and Pull Requests has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:17:52</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re currently working on mitigating elevated errors and intermittent failures affecting GitHub, including Pull Requests, Git Operations, and Issues. Some requests may fail or be delayed while we work to restore service. We&#039;ll provide another update as more information becomes available..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>11:11:30</var> GMT+0</small><br><strong>Investigating</strong> -
API Requests is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>10:38:49</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re investigating elevated errors and intermittent failures affecting GitHub including Pull Requests, Git Operations, and Issues. We&#039;ll continue to keep users updated as we work toward mitigation..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>09:53:38</var> GMT+0</small><br><strong>Investigating</strong> -
Pull Requests is experiencing degraded availability. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>09:50:47</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>09:49:55</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Git Operations, Issues and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Wed, 4 Nov 2026 09:49:55 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw1xwc000yu60rofi7yazw9t</link>
<guid>https://status.ubicloud.com/incident/cmw1xwc000yu60rofi7yazw9t</guid>
</item>
<item>
<title>Network connectivity issues in Virgina, US</title>
<description>
Type: Incident
Affected Components: PostgreSQL, Networking, Compute
Nov 4, 08:00:00 GMT+0 - Resolved - This incident has been resolved.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved..</p>
]]>
</content:encoded>
<pubDate>Wed, 4 Nov 2026 08:00:00 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw1stz3906le13pk3jojpaq1</link>
<guid>https://status.ubicloud.com/incident/cmw1stz3906le13pk3jojpaq1</guid>
</item>
<item>
<title>Incident with API, Git, and Copilot requests</title>
<description>
Type: Incident
Duration: 1 day, 6 hours and 59 minutes
Affected Components: GitHub → API Requests
Nov 3, 13:56:18 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Nov 3, 13:54:20 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Nov 3, 13:52:21 GMT+0 - Investigating - API Requests is operating normally. Nov 3, 13:51:46 GMT+0 - Monitoring - The degradation affecting Copilot has been mitigated. We are monitoring to ensure stability.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 6 hours and 59 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:56:18</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:54:20</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:52:21</var> GMT+0</small><br><strong>Investigating</strong> -
API Requests is operating normally..</p>
<p><small>Nov <var data-var='date'> 3</var>, <var data-var='time'>13:51:46</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Copilot has been mitigated. We are monitoring to ensure stability..</p>
]]>
</content:encoded>
<pubDate>Tue, 3 Nov 2026 13:25:19 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmw0r14r600lz0krtcvtlmjoo</link>
<guid>https://status.ubicloud.com/incident/cmw0r14r600lz0krtcvtlmjoo</guid>
</item>
<item>
<title>Incident with Actions, Pages, and Webhooks</title>
<description>
Type: Incident
Duration: 3 days, 20 hours and 45 minutes
Affected Components: GitHub → Webhooks, GitHub → Actions
Oct 30, 19:23:40 GMT+0 - Resolved - On October 30, 2026, from 17:50 to 18:32 UTC, Actions, Pages, and Webhooks experienced degraded performance. Actions workflow processing, Pages deployments, and webhook deliveries were delayed, and code search and live updates returned errors. Some Actions queues took additional time to drain after mitigation, and the incident was resolved at 19:23 UTC. &lt;br /&gt;&lt;br /&gt;The impact was triggered by maintenance on a cloud network gateway connecting an Actions service in the GitHub data center to Redis clusters in the Central US Azure region. The service&#039;s Redis client retried aggressively and redialed those connections. That produced a connection storm that saturated the gateway, with about 90% of connection attempts failing. Other services sharing the same network path were degraded as well. &lt;br /&gt;&lt;br /&gt;We mitigated the incident by removing the affected resources from rotation, which ended the connection storm and allowed the gateway to recover. We have already reduced the number of connection retries in the impacted service and are further improving retry behavior with jittered backoff, shorter connect timeouts, and retry budgets. We are also working with our infrastructure team to make GitHub more resilient to gateway maintenance. Oct 30, 18:32:39 GMT+0 - Monitoring - The degradation affecting Actions, Pages and Webhooks has been mitigated. We are monitoring to ensure stability. Oct 30, 18:32:23 GMT+0 - Investigating - All affected services have recovered following mitigation. We continue to investigate root cause. Oct 30, 18:06:21 GMT+0 - Investigating - We continue to investigate delays affecting Actions workflows, Pages deployments, and webhook events, along with errors affecting code search and live updates. We have taken mitigation steps. Actions processing and Pages deployments are showing signs of recovery, but impact has not fully resolved. We are monitoring recovery and investigating the underlying cause. Oct 30, 18:00:58 GMT+0 - Investigating - Pages is experiencing degraded performance. We are continuing to investigate. Oct 30, 17:50:54 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Webhooks
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 3 days, 20 hours and 45 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>19:23:40</var> GMT+0</small><br><strong>Resolved</strong> -
On October 30, 2026, from 17:50 to 18:32 UTC, Actions, Pages, and Webhooks experienced degraded performance. Actions workflow processing, Pages deployments, and webhook deliveries were delayed, and code search and live updates returned errors. Some Actions queues took additional time to drain after mitigation, and the incident was resolved at 19:23 UTC. &lt;br /&gt;&lt;br /&gt;The impact was triggered by maintenance on a cloud network gateway connecting an Actions service in the GitHub data center to Redis clusters in the Central US Azure region. The service&#039;s Redis client retried aggressively and redialed those connections. That produced a connection storm that saturated the gateway, with about 90% of connection attempts failing. Other services sharing the same network path were degraded as well. &lt;br /&gt;&lt;br /&gt;We mitigated the incident by removing the affected resources from rotation, which ended the connection storm and allowed the gateway to recover. We have already reduced the number of connection retries in the impacted service and are further improving retry behavior with jittered backoff, shorter connect timeouts, and retry budgets. We are also working with our infrastructure team to make GitHub more resilient to gateway maintenance..</p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>18:32:39</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions, Pages and Webhooks has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>18:32:23</var> GMT+0</small><br><strong>Investigating</strong> -
All affected services have recovered following mitigation. We continue to investigate root cause..</p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>18:06:21</var> GMT+0</small><br><strong>Investigating</strong> -
We continue to investigate delays affecting Actions workflows, Pages deployments, and webhook events, along with errors affecting code search and live updates. We have taken mitigation steps. Actions processing and Pages deployments are showing signs of recovery, but impact has not fully resolved. We are monitoring recovery and investigating the underlying cause..</p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>18:00:58</var> GMT+0</small><br><strong>Investigating</strong> -
Pages is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>17:50:54</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Webhooks.</p>
]]>
</content:encoded>
<pubDate>Fri, 30 Oct 2026 17:50:54 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvv9ud1g1i080rp2mxkdw68p</link>
<guid>https://status.ubicloud.com/incident/cmvv9ud1g1i080rp2mxkdw68p</guid>
</item>
<item>
<title>Increased provisioning times for GitHub runners</title>
<description>
Type: Incident
Duration: 1 hour and 57 minutes
Affected Components: Ubicloud Runners
Oct 30, 13:17:38 GMT+0 - Resolved - We investigated the cache slowness. On October 29, Cloudflare had issues on several of their services. From approximately 09:00 to 22:00 UTC, Cloudflare sent our cache traffic to its Prague location instead of Frankfurt, and Cloudflare R2 had increased latency. Because of this, cache upload and download speeds were much lower than normal.
Cache performance has been normal since approximately 23:00 UTC on October 29.
We are working with Cloudflare to find the root cause, and to prevent and detect similar problems earlier in the future. Oct 29, 19:17:27 GMT+0 - Resolved - Wait times for GitHub Actions runners are back to normal.
Cause: Today we saw a surge in demand that put pressure on our capacity at peak times, especially while EU evenings and US mornings overlapped. This week the overlap is an hour longer than usual because Europe has already switched to winter time and the US hasn&#039;t yet. Slowness in Cloudflare R2, which we use for cache storage, also made jobs run longer. This further added to queue times.
We&#039;re adding capacity to handle a continuous increase in overall demand. We apologize for the delay in processing times. Oct 29, 18:23:28 GMT+0 - Monitoring - Wait times are improving. We&#039;re processing the remaining backlog and monitoring until queue times are back to normal. Oct 29, 17:20:35 GMT+0 - Investigating - We&#039;re seeing longer than usual wait times for GitHub Actions runners to start, due to high demand. We&#039;re investigating and will post updates here.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 hour and 57 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 30</var>, <var data-var='time'>13:17:38</var> GMT+0</small><br><strong>Resolved</strong> -
We investigated the cache slowness. On October 29, Cloudflare had issues on several of their services. From approximately 09:00 to 22:00 UTC, Cloudflare sent our cache traffic to its Prague location instead of Frankfurt, and Cloudflare R2 had increased latency. Because of this, cache upload and download speeds were much lower than normal.
Cache performance has been normal since approximately 23:00 UTC on October 29.
We are working with Cloudflare to find the root cause, and to prevent and detect similar problems earlier in the future..</p>
<p><small>Oct <var data-var='date'> 29</var>, <var data-var='time'>19:17:27</var> GMT+0</small><br><strong>Resolved</strong> -
Wait times for GitHub Actions runners are back to normal.
Cause: Today we saw a surge in demand that put pressure on our capacity at peak times, especially while EU evenings and US mornings overlapped. This week the overlap is an hour longer than usual because Europe has already switched to winter time and the US hasn&#039;t yet. Slowness in Cloudflare R2, which we use for cache storage, also made jobs run longer. This further added to queue times.
We&#039;re adding capacity to handle a continuous increase in overall demand. We apologize for the delay in processing times..</p>
<p><small>Oct <var data-var='date'> 29</var>, <var data-var='time'>18:23:28</var> GMT+0</small><br><strong>Monitoring</strong> -
Wait times are improving. We&#039;re processing the remaining backlog and monitoring until queue times are back to normal..</p>
<p><small>Oct <var data-var='date'> 29</var>, <var data-var='time'>17:20:35</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re seeing longer than usual wait times for GitHub Actions runners to start, due to high demand. We&#039;re investigating and will post updates here..</p>
]]>
</content:encoded>
<pubDate>Thu, 29 Oct 2026 17:20:35 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvtt7f1k02sm1mphc3uplj42</link>
<guid>https://status.ubicloud.com/incident/cmvtt7f1k02sm1mphc3uplj42</guid>
</item>
<item>
<title>API is back up</title>
<description>
Type: Incident
Affected Components: API
Oct 28, 01:02:25 GMT+0 - Resolved - API is back up. This incident was automatically resolved by Instatus monitoring. Oct 28, 01:01:52 GMT+0 - Investigating - API is down at the moment. This incident was automatically created by Instatus monitoring.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 28</var>, <var data-var='time'>01:02:25</var> GMT+0</small><br><strong>Resolved</strong> -
API is back up. This incident was automatically resolved by Instatus monitoring..</p>
<p><small>Oct <var data-var='date'> 28</var>, <var data-var='time'>01:01:52</var> GMT+0</small><br><strong>Investigating</strong> -
API is down at the moment. This incident was automatically created by Instatus monitoring..</p>
]]>
</content:encoded>
<pubDate>Wed, 28 Oct 2026 01:01:52 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvreswkz004x07nqbibxgjpf</link>
<guid>https://status.ubicloud.com/incident/cmvreswkz004x07nqbibxgjpf</guid>
</item>
<item>
<title>Console is back up</title>
<description>
Type: Incident
Affected Components: Console
Oct 28, 01:02:31 GMT+0 - Resolved - Console is back up. This incident was automatically resolved by Instatus monitoring. Oct 28, 01:01:51 GMT+0 - Investigating - Console is down at the moment. This incident was automatically created by Instatus monitoring.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 28</var>, <var data-var='time'>01:02:31</var> GMT+0</small><br><strong>Resolved</strong> -
Console is back up. This incident was automatically resolved by Instatus monitoring..</p>
<p><small>Oct <var data-var='date'> 28</var>, <var data-var='time'>01:01:51</var> GMT+0</small><br><strong>Investigating</strong> -
Console is down at the moment. This incident was automatically created by Instatus monitoring..</p>
]]>
</content:encoded>
<pubDate>Wed, 28 Oct 2026 01:01:51 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvresy6j006w07o9m25m1q6f</link>
<guid>https://status.ubicloud.com/incident/cmvresy6j006w07o9m25m1q6f</guid>
</item>
<item>
<title>Incident with Actions, Issues, API Requests and Pull Requests</title>
<description>
Type: Incident
Duration: 8 hours and 56 minutes
Affected Components: GitHub → API Requests, GitHub → Actions
Oct 26, 18:57:22 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Oct 26, 18:54:04 GMT+0 - Monitoring - We saw elevated latency affecting REST API requests, including Checks, and some Actions, Issues, Pull Requests, and GraphQL functionality. Some requests may have been slow or timed out, and notifications were delayed from 17:59 UTC - 18:40 UTC.&lt;br /&gt;&lt;br /&gt;We mitigated the issue and all services have recovered. Oct 26, 18:50:58 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Oct 26, 18:48:26 GMT+0 - Investigating - We are investigating reports of degraded performance for API Requests, Actions, Issues and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 8 hours and 56 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 26</var>, <var data-var='time'>18:57:22</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Oct <var data-var='date'> 26</var>, <var data-var='time'>18:54:04</var> GMT+0</small><br><strong>Monitoring</strong> -
We saw elevated latency affecting REST API requests, including Checks, and some Actions, Issues, Pull Requests, and GraphQL functionality. Some requests may have been slow or timed out, and notifications were delayed from 17:59 UTC - 18:40 UTC.&lt;br /&gt;&lt;br /&gt;We mitigated the issue and all services have recovered..</p>
<p><small>Oct <var data-var='date'> 26</var>, <var data-var='time'>18:50:58</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 26</var>, <var data-var='time'>18:48:26</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for API Requests, Actions, Issues and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Mon, 26 Oct 2026 18:48:26 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvpm2513044p0rnr0t0v76xf</link>
<guid>https://status.ubicloud.com/incident/cmvpm2513044p0rnr0t0v76xf</guid>
</item>
<item>
<title>Delayed Actions status check results</title>
<description>
Type: Incident
Duration: 1 day, 12 hours and 24 minutes
Affected Components: GitHub → Actions
Oct 23, 08:05:27 GMT+0 - Resolved - On October 23, 2026, between 07:06 UTC and 07:27 UTC, GitHub Actions workflow run results were delayed. Job status, check run updates, and logs in the GitHub UI and API updated with an average delay of about 2 minutes, and some updates were delayed by up to about 7 minutes. &lt;br /&gt;&lt;br /&gt;During this window, a storage dependency that Actions uses to process workflow logs and artifacts throttled our requests. This slowed result processing, causing updates to queue up. The queue was fully processed by 07:27 UTC, shortly after the throttling stopped. &lt;br /&gt;&lt;br /&gt;To reduce the risk of recurrence, we have added several storage accounts to increase capacity, and we are working on additional steps to continue to scale. We are also working with our storage provider to understand why the throttling occurred, and improving how result processing handles storage throttling and how quickly we detect it. Oct 23, 07:56:35 GMT+0 - Monitoring - As of 07:27 UTC, Actions results are delivering within normal SLOs again.&lt;br /&gt;&lt;br /&gt;Thank you for this patience during this period of delayed Actions results delivery. Our team continues to monitor system health and explore refinements to reduce impact to customers in the future. Oct 23, 07:53:40 GMT+0 - Monitoring - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Oct 23, 07:31:54 GMT+0 - Investigating - We are seeing delays in results from Actions workflows being reported within GitHub starting from 07:07 UTC.&lt;br /&gt;&lt;br /&gt;Recovery is already being seen but our team is continuing to investigate and monitor performance. Oct 23, 07:29:03 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 12 hours and 24 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>08:05:27</var> GMT+0</small><br><strong>Resolved</strong> -
On October 23, 2026, between 07:06 UTC and 07:27 UTC, GitHub Actions workflow run results were delayed. Job status, check run updates, and logs in the GitHub UI and API updated with an average delay of about 2 minutes, and some updates were delayed by up to about 7 minutes. &lt;br /&gt;&lt;br /&gt;During this window, a storage dependency that Actions uses to process workflow logs and artifacts throttled our requests. This slowed result processing, causing updates to queue up. The queue was fully processed by 07:27 UTC, shortly after the throttling stopped. &lt;br /&gt;&lt;br /&gt;To reduce the risk of recurrence, we have added several storage accounts to increase capacity, and we are working on additional steps to continue to scale. We are also working with our storage provider to understand why the throttling occurred, and improving how result processing handles storage throttling and how quickly we detect it..</p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>07:56:35</var> GMT+0</small><br><strong>Monitoring</strong> -
As of 07:27 UTC, Actions results are delivering within normal SLOs again.&lt;br /&gt;&lt;br /&gt;Thank you for this patience during this period of delayed Actions results delivery. Our team continues to monitor system health and explore refinements to reduce impact to customers in the future..</p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>07:53:40</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>07:31:54</var> GMT+0</small><br><strong>Investigating</strong> -
We are seeing delays in results from Actions workflows being reported within GitHub starting from 07:07 UTC.&lt;br /&gt;&lt;br /&gt;Recovery is already being seen but our team is continuing to investigate and monitor performance..</p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>07:29:03</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions.</p>
]]>
</content:encoded>
<pubDate>Fri, 23 Oct 2026 07:29:03 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvknm0xe13660ro2mjyr3c5w</link>
<guid>https://status.ubicloud.com/incident/cmvknm0xe13660ro2mjyr3c5w</guid>
</item>
<item>
<title>Project updates delayed</title>
<description>
Type: Incident
Duration: 27 days, 14 hours and 21 minutes
Affected Components: GitHub → Actions, GitHub → API Requests
Oct 23, 02:30:25 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Oct 23, 00:55:18 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Oct 22, 23:48:05 GMT+0 - Investigating - GitHub Projects update processing has improved, but newly added items may still take longer than expected to appear. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to work on the remaining delays and monitor recovery. Oct 22, 22:07:36 GMT+0 - Investigating - Some GitHub Projects updates, including newly added items and changes to issues and pull requests, are still delayed.&lt;br /&gt;&lt;br /&gt;We have seen some recovery, and are working to add more capacity to accelerate the recovery. Oct 22, 20:46:26 GMT+0 - Investigating - Some GitHub Projects updates, including newly added items and changes to issues and pull requests, remain delayed.&lt;br /&gt;&lt;br /&gt;We&#039;ve identified interruptions in update processing and are evaluating mitigations to restore normal operation. Oct 22, 19:56:09 GMT+0 - Investigating - Some GitHub Projects updates related to issue changes and merged pull requests are still delayed. We&#039;re investigating the remaining delays and working to restore normal processing. Oct 22, 19:04:03 GMT+0 - Investigating - Some updates in GitHub Projects are still delayed. We&#039;re continuing mitigation work and monitoring. Oct 22, 18:26:31 GMT+0 - Investigating - The other affected services have recovered. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to investigate GitHub Projects and verify that normal operation has been restored. Oct 22, 17:43:44 GMT+0 - Investigating - We&#039;re seeing recovery across the affected services. GitHub Projects is still recovering as queued updates are processed, and we&#039;re continuing to monitor for stability. Oct 22, 17:41:38 GMT+0 - Investigating - Actions, Issues, Packages and Pull Requests are operating normally. Oct 22, 17:16:06 GMT+0 - Investigating - We&#039;re seeing recovery after adding capacity, with errors decreasing. Some users may still experience errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to work towards full recovery and monitoring for stability. Oct 22, 16:41:52 GMT+0 - Investigating - Some users continue to experience errors loading Issues, Pull Requests, package listings and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re working through multiple mitigation efforts and continuing to investigate. Oct 22, 16:35:07 GMT+0 - Investigating - API Requests is operating normally. Oct 22, 16:28:56 GMT+0 - Investigating - Packages is experiencing degraded performance. We are continuing to investigate. Oct 22, 16:26:16 GMT+0 - Investigating - API Requests is experiencing degraded performance. We are continuing to investigate. Oct 22, 16:08:04 GMT+0 - Investigating - Some users continue to experience intermittent errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re investigating and evaluating mitigation options to restore normal service. Oct 22, 15:33:52 GMT+0 - Investigating - Some users may experience intermittent errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re investigating and working to restore normal service. Oct 22, 15:28:04 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions, Issues and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 27 days, 14 hours and 21 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>02:30:25</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Oct <var data-var='date'> 23</var>, <var data-var='time'>00:55:18</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>23:48:05</var> GMT+0</small><br><strong>Investigating</strong> -
GitHub Projects update processing has improved, but newly added items may still take longer than expected to appear. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to work on the remaining delays and monitor recovery..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>22:07:36</var> GMT+0</small><br><strong>Investigating</strong> -
Some GitHub Projects updates, including newly added items and changes to issues and pull requests, are still delayed.&lt;br /&gt;&lt;br /&gt;We have seen some recovery, and are working to add more capacity to accelerate the recovery..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>20:46:26</var> GMT+0</small><br><strong>Investigating</strong> -
Some GitHub Projects updates, including newly added items and changes to issues and pull requests, remain delayed.&lt;br /&gt;&lt;br /&gt;We&#039;ve identified interruptions in update processing and are evaluating mitigations to restore normal operation..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>19:56:09</var> GMT+0</small><br><strong>Investigating</strong> -
Some GitHub Projects updates related to issue changes and merged pull requests are still delayed. We&#039;re investigating the remaining delays and working to restore normal processing..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>19:04:03</var> GMT+0</small><br><strong>Investigating</strong> -
Some updates in GitHub Projects are still delayed. We&#039;re continuing mitigation work and monitoring..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>18:26:31</var> GMT+0</small><br><strong>Investigating</strong> -
The other affected services have recovered. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to investigate GitHub Projects and verify that normal operation has been restored..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>17:43:44</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re seeing recovery across the affected services. GitHub Projects is still recovering as queued updates are processed, and we&#039;re continuing to monitor for stability..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>17:41:38</var> GMT+0</small><br><strong>Investigating</strong> -
Actions, Issues, Packages and Pull Requests are operating normally..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>17:16:06</var> GMT+0</small><br><strong>Investigating</strong> -
We&#039;re seeing recovery after adding capacity, with errors decreasing. Some users may still experience errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re continuing to work towards full recovery and monitoring for stability..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>16:41:52</var> GMT+0</small><br><strong>Investigating</strong> -
Some users continue to experience errors loading Issues, Pull Requests, package listings and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re working through multiple mitigation efforts and continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>16:35:07</var> GMT+0</small><br><strong>Investigating</strong> -
API Requests is operating normally..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>16:28:56</var> GMT+0</small><br><strong>Investigating</strong> -
Packages is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>16:26:16</var> GMT+0</small><br><strong>Investigating</strong> -
API Requests is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>16:08:04</var> GMT+0</small><br><strong>Investigating</strong> -
Some users continue to experience intermittent errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re investigating and evaluating mitigation options to restore normal service..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:33:52</var> GMT+0</small><br><strong>Investigating</strong> -
Some users may experience intermittent errors loading Issues, Pull Requests and search results on github.com. &lt;br /&gt;&lt;br /&gt;We&#039;re investigating and working to restore normal service..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:28:04</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions, Issues and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Thu, 22 Oct 2026 15:28:03 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvjp54y30jct0rp4lsyb0961</link>
<guid>https://status.ubicloud.com/incident/cmvjp54y30jct0rp4lsyb0961</guid>
</item>
<item>
<title>Incident with Actions</title>
<description>
Type: Incident
Duration: 2 days, 12 hours and 30 minutes
Affected Components: GitHub → Actions
Oct 22, 15:18:46 GMT+0 - Resolved - On October 22, 2026, from 13:05 to 15:20 UTC, our event streaming platform was degraded resulting in elevated job processing latency spikes of 19-35 seconds and elevated job processing times of 4m – this resulted in delayed Actions workflow and step results (up to 13m), notifications (20m) and repository push events (12m). Recovery for repositories began at 14:32, with Actions recovery beginning at 14:59. The issue reoccurred at 16:26 again impacting Actions runs, repo push events, and notifications. Recovery began at 17:16 and completed at 2:07. &lt;br /&gt;&lt;br /&gt;The incident was caused by a significant increase in load coupled with client connection storms during a deployment overloading our Kafka event streaming brokers. &lt;br /&gt;&lt;br /&gt;To prevent future recurrence, we have expanded the affected cluster, shifted traffic to less loaded clusters, and implemented tunable client reconnect backoff. Oct 22, 15:18:41 GMT+0 - Monitoring - GitHub Actions workflow and step result processing has recovered. Repository update processing has also returned to normal. Oct 22, 15:17:23 GMT+0 - Monitoring - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Oct 22, 15:06:35 GMT+0 - Investigating - We’re seeing recovery, and repository update processing has returned to normal. Delays affecting GitHub Actions workflow and step results are continuing to clear as we scale processing capacity; the remaining backlog is expected to take about 20 minutes to drain. Oct 22, 14:35:29 GMT+0 - Investigating - Repository update processing has recovered. We’re continuing to work on delays affecting GitHub Actions workflow and step results. We’ll share another update as we make progress. Oct 22, 14:19:32 GMT+0 - Investigating - We’re investigating delays in processing GitHub Actions workflow and step results, along with some repository updates. Oct 22, 14:18:15 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 2 days, 12 hours and 30 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:18:46</var> GMT+0</small><br><strong>Resolved</strong> -
On October 22, 2026, from 13:05 to 15:20 UTC, our event streaming platform was degraded resulting in elevated job processing latency spikes of 19-35 seconds and elevated job processing times of 4m – this resulted in delayed Actions workflow and step results (up to 13m), notifications (20m) and repository push events (12m). Recovery for repositories began at 14:32, with Actions recovery beginning at 14:59. The issue reoccurred at 16:26 again impacting Actions runs, repo push events, and notifications. Recovery began at 17:16 and completed at 2:07. &lt;br /&gt;&lt;br /&gt;The incident was caused by a significant increase in load coupled with client connection storms during a deployment overloading our Kafka event streaming brokers. &lt;br /&gt;&lt;br /&gt;To prevent future recurrence, we have expanded the affected cluster, shifted traffic to less loaded clusters, and implemented tunable client reconnect backoff..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:18:41</var> GMT+0</small><br><strong>Monitoring</strong> -
GitHub Actions workflow and step result processing has recovered. Repository update processing has also returned to normal..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:17:23</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>15:06:35</var> GMT+0</small><br><strong>Investigating</strong> -
We’re seeing recovery, and repository update processing has returned to normal. Delays affecting GitHub Actions workflow and step results are continuing to clear as we scale processing capacity; the remaining backlog is expected to take about 20 minutes to drain..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>14:35:29</var> GMT+0</small><br><strong>Investigating</strong> -
Repository update processing has recovered. We’re continuing to work on delays affecting GitHub Actions workflow and step results. We’ll share another update as we make progress..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>14:19:32</var> GMT+0</small><br><strong>Investigating</strong> -
We’re investigating delays in processing GitHub Actions workflow and step results, along with some repository updates..</p>
<p><small>Oct <var data-var='date'> 22</var>, <var data-var='time'>14:18:15</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions.</p>
]]>
</content:encoded>
<pubDate>Thu, 22 Oct 2026 14:18:15 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvjmn0go0ep80rp4s741y6ep</link>
<guid>https://status.ubicloud.com/incident/cmvjmn0go0ep80rp4s741y6ep</guid>
</item>
<item>
<title>Delays for Pull Requests and Webhook deliveries</title>
<description>
Type: Incident
Duration: 11 days, 6 hours and 2 minutes
Affected Components: GitHub → Webhooks, GitHub → Actions
Oct 21, 19:22:52 GMT+0 - Resolved - This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available. Oct 21, 19:22:38 GMT+0 - Investigating - Pull request updates, Actions, and webhook deliveries have recovered. Audit log delays are now below 40 minutes, within the healthy threshold, and continue to decrease as the remaining backlog is processed. Oct 21, 18:52:50 GMT+0 - Investigating - Audit log events continue to recover, with delays now below 70 minutes. We’ve applied mitigations to speed recovery and expect audit logs to fully recover in about 40 minutes. Pull request updates, Actions, and webhook deliveries remain recovered. We are monitoring progress and will share further updates. Oct 21, 18:18:11 GMT+0 - Investigating - Audit log events are recovering, with delays now below 90 minutes as the remaining backlog is processed. We’ve reduced load on the event-streaming platform and are monitoring recovery. Further updates will follow. Oct 21, 17:09:35 GMT+0 - Investigating - Pull request updates, Actions, and webhook deliveries have recovered. Audit log events are recovering and delays are decreasing, but we expect the remaining backlog to take approximately two to three hours to clear. We’re monitoring recovery and will share further updates. Oct 21, 16:59:45 GMT+0 - Investigating - Actions and webhook deliveries have recovered. Audit log events are still delayed by about 100 minutes. We are working through the backlog, and full recovery will take some time. We will share more updates as recovery continues. Oct 21, 16:57:42 GMT+0 - Investigating - Actions and Webhooks are operating normally. Oct 21, 16:34:49 GMT+0 - Investigating - Audit log events continue to be delayed by up to 110 minutes. Pull request updates have recovered. Actions is recovering, and users should no longer experience impact in Actions. Webhook deliveries continue to recover as we work through backlogs. We are monitoring recovery and will share further updates as we learn more. Oct 21, 16:33:10 GMT+0 - Investigating - Pull Requests is operating normally. Oct 21, 16:25:30 GMT+0 - Investigating - Audit log events are delayed by up to 100 minutes, and Actions workflow status updates continue to experience extended delays. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring recovery and will share further updates as we learn more. Oct 21, 16:17:23 GMT+0 - Investigating - Audit log events and Actions workflow status updates continue to be delayed. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring the recovery and will share further updates as we learn more. Oct 21, 15:58:38 GMT+0 - Investigating - We continue to see delays affecting pull request updates, webhook deliveries, and audit log events. The event streaming platform is showing signs of recovery, and we are monitoring as the impact continues. Oct 21, 15:36:47 GMT+0 - Investigating - We are investigating delays affecting pull request updates and webhook deliveries. Increased load on our event streaming platform is contributing to hotspots. We are shifting traffic away from affected areas and adding processing capacity; impact is ongoing while these mitigations take effect. Oct 21, 15:15:04 GMT+0 - Investigating - Actions is experiencing degraded performance. We are continuing to investigate. Oct 21, 14:52:50 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions, Pull Requests and Webhooks
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 11 days, 6 hours and 2 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>19:22:52</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>19:22:38</var> GMT+0</small><br><strong>Investigating</strong> -
Pull request updates, Actions, and webhook deliveries have recovered. Audit log delays are now below 40 minutes, within the healthy threshold, and continue to decrease as the remaining backlog is processed..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>18:52:50</var> GMT+0</small><br><strong>Investigating</strong> -
Audit log events continue to recover, with delays now below 70 minutes. We’ve applied mitigations to speed recovery and expect audit logs to fully recover in about 40 minutes. Pull request updates, Actions, and webhook deliveries remain recovered. We are monitoring progress and will share further updates..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>18:18:11</var> GMT+0</small><br><strong>Investigating</strong> -
Audit log events are recovering, with delays now below 90 minutes as the remaining backlog is processed. We’ve reduced load on the event-streaming platform and are monitoring recovery. Further updates will follow..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>17:09:35</var> GMT+0</small><br><strong>Investigating</strong> -
Pull request updates, Actions, and webhook deliveries have recovered. Audit log events are recovering and delays are decreasing, but we expect the remaining backlog to take approximately two to three hours to clear. We’re monitoring recovery and will share further updates..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:59:45</var> GMT+0</small><br><strong>Investigating</strong> -
Actions and webhook deliveries have recovered. Audit log events are still delayed by about 100 minutes. We are working through the backlog, and full recovery will take some time. We will share more updates as recovery continues..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:57:42</var> GMT+0</small><br><strong>Investigating</strong> -
Actions and Webhooks are operating normally..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:34:49</var> GMT+0</small><br><strong>Investigating</strong> -
Audit log events continue to be delayed by up to 110 minutes. Pull request updates have recovered. Actions is recovering, and users should no longer experience impact in Actions. Webhook deliveries continue to recover as we work through backlogs. We are monitoring recovery and will share further updates as we learn more..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:33:10</var> GMT+0</small><br><strong>Investigating</strong> -
Pull Requests is operating normally..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:25:30</var> GMT+0</small><br><strong>Investigating</strong> -
Audit log events are delayed by up to 100 minutes, and Actions workflow status updates continue to experience extended delays. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring recovery and will share further updates as we learn more..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>16:17:23</var> GMT+0</small><br><strong>Investigating</strong> -
Audit log events and Actions workflow status updates continue to be delayed. Pull request updates and webhook deliveries are recovering as we restore event processing and work through backlogs. We’re monitoring the recovery and will share further updates as we learn more..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>15:58:38</var> GMT+0</small><br><strong>Investigating</strong> -
We continue to see delays affecting pull request updates, webhook deliveries, and audit log events. The event streaming platform is showing signs of recovery, and we are monitoring as the impact continues..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>15:36:47</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating delays affecting pull request updates and webhook deliveries. Increased load on our event streaming platform is contributing to hotspots. We are shifting traffic away from affected areas and adding processing capacity; impact is ongoing while these mitigations take effect..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>15:15:04</var> GMT+0</small><br><strong>Investigating</strong> -
Actions is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>14:52:50</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions, Pull Requests and Webhooks.</p>
]]>
</content:encoded>
<pubDate>Wed, 21 Oct 2026 14:52:49 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvi8indy0ib70rqtuspa54yo</link>
<guid>https://status.ubicloud.com/incident/cmvi8indy0ib70rqtuspa54yo</guid>
</item>
<item>
<title>Degraded availability for GitHub Actions, GitHub Pages and Webhooks</title>
<description>
Type: Incident
Duration: 2 days, 19 hours and 32 minutes
Affected Components: GitHub → Webhooks, Ubicloud Runners, GitHub → Actions
Oct 19, 17:44:18 GMT+0 - Resolved - Following GitHub’s earlier incident, the queued jobs released at once created a backlog on our runners. That backlog has now cleared, and jobs are being picked up normally. We’ll continue to monitor. Thank you for your patience. Oct 19, 17:14:12 GMT+0 - Monitoring - GitHub had a major incident that delayed webhook delivery. They have resolved the issue, and we are able to provision runners successfully. Since they released the queued jobs all at once, there might be some delays in processing jobs due to the backlog. Oct 19, 16:56:58 GMT+0 - Resolved - On October 19, 2026, between 15:24 and 16:52 UTC, GitHub Actions, GitHub Pages, and webhooks generated by Actions were degraded. Live updates on GitHub.com and code search were also degraded for part of this time. No data was lost. &lt;br /&gt;&lt;br /&gt;Impact &lt;br /&gt;&lt;br /&gt; - Workflow status updates, logs, and checks were delayed. &lt;br /&gt; - 5.13% of jobs on GitHub-hosted runners were delayed in starting, and 0.02% of workflow runs failed. &lt;br /&gt; - During the first 30 minutes, 8.47% of requests to the Actions jobs REST API returned errors. &lt;br /&gt;&lt;br /&gt;Pages deployments and webhooks generated by Actions runs were delayed. &lt;br /&gt;&lt;br /&gt;An internal Actions service that stores logs and run status lost connections to some of its cache servers in one cloud region and immediately retried them from every server at once. The resulting surge of new connections overloaded a shared network link between our data center and that region, which also disrupted other services that use the same link. We mitigated the incident by moving the Actions service off the affected cache servers and reducing connections from other services on the link. We then processed the delayed Actions work. We are continuing to investigate the cause of the initial connection timeouts. &lt;br /&gt;&lt;br /&gt;We were already in the process of rolling out a limit on how quickly this service can open new connections, but it was still in shadow mode (logging only, not enforcing) at the time of the incident. We have since fully enforced it and capped the service&#039;s concurrent connection attempts. We are extending these limits to other Actions services, adding backoff to retries, limiting how much shared network capacity a single service can use, and improving alerting so we can find the source of connection surges faster. Oct 19, 16:56:54 GMT+0 - Monitoring - Actions, Pages and Webhooks are operating normally. Oct 19, 16:30:29 GMT+0 - Investigating - We’re seeing recovery following mitigation and are continuing to monitor. &lt;br /&gt;&lt;br /&gt;Customers may still experience delays in GitHub Actions workflow status updates, GitHub Pages deployments, and check and workflow webhook events generated by Actions runs. Oct 19, 15:56:34 GMT+0 - Investigating - We’re currently investigating degraded availability affecting GitHub Actions, GitHub Pages and Webhooks. &lt;br /&gt;&lt;br /&gt;Customers may experience delays in workflow processing, site builds and webhook delivery.&lt;br /&gt; Oct 19, 15:50:10 GMT+0 - Investigating - Webhooks is experiencing degraded performance. We are continuing to investigate. Oct 19, 15:49:25 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions and Pages
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 2 days, 19 hours and 32 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>17:44:18</var> GMT+0</small><br><strong>Resolved</strong> -
Following GitHub’s earlier incident, the queued jobs released at once created a backlog on our runners. That backlog has now cleared, and jobs are being picked up normally. We’ll continue to monitor. Thank you for your patience..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>17:14:12</var> GMT+0</small><br><strong>Monitoring</strong> -
GitHub had a major incident that delayed webhook delivery. They have resolved the issue, and we are able to provision runners successfully. Since they released the queued jobs all at once, there might be some delays in processing jobs due to the backlog..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>16:56:58</var> GMT+0</small><br><strong>Resolved</strong> -
On October 19, 2026, between 15:24 and 16:52 UTC, GitHub Actions, GitHub Pages, and webhooks generated by Actions were degraded. Live updates on GitHub.com and code search were also degraded for part of this time. No data was lost. &lt;br /&gt;&lt;br /&gt;Impact &lt;br /&gt;&lt;br /&gt; - Workflow status updates, logs, and checks were delayed. &lt;br /&gt; - 5.13% of jobs on GitHub-hosted runners were delayed in starting, and 0.02% of workflow runs failed. &lt;br /&gt; - During the first 30 minutes, 8.47% of requests to the Actions jobs REST API returned errors. &lt;br /&gt;&lt;br /&gt;Pages deployments and webhooks generated by Actions runs were delayed. &lt;br /&gt;&lt;br /&gt;An internal Actions service that stores logs and run status lost connections to some of its cache servers in one cloud region and immediately retried them from every server at once. The resulting surge of new connections overloaded a shared network link between our data center and that region, which also disrupted other services that use the same link. We mitigated the incident by moving the Actions service off the affected cache servers and reducing connections from other services on the link. We then processed the delayed Actions work. We are continuing to investigate the cause of the initial connection timeouts. &lt;br /&gt;&lt;br /&gt;We were already in the process of rolling out a limit on how quickly this service can open new connections, but it was still in shadow mode (logging only, not enforcing) at the time of the incident. We have since fully enforced it and capped the service&#039;s concurrent connection attempts. We are extending these limits to other Actions services, adding backoff to retries, limiting how much shared network capacity a single service can use, and improving alerting so we can find the source of connection surges faster..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>16:56:54</var> GMT+0</small><br><strong>Monitoring</strong> -
Actions, Pages and Webhooks are operating normally..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>16:30:29</var> GMT+0</small><br><strong>Investigating</strong> -
We’re seeing recovery following mitigation and are continuing to monitor. &lt;br /&gt;&lt;br /&gt;Customers may still experience delays in GitHub Actions workflow status updates, GitHub Pages deployments, and check and workflow webhook events generated by Actions runs..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>15:56:34</var> GMT+0</small><br><strong>Investigating</strong> -
We’re currently investigating degraded availability affecting GitHub Actions, GitHub Pages and Webhooks. &lt;br /&gt;&lt;br /&gt;Customers may experience delays in workflow processing, site builds and webhook delivery.&lt;br /&gt;.</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>15:50:10</var> GMT+0</small><br><strong>Investigating</strong> -
Webhooks is experiencing degraded performance. We are continuing to investigate..</p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>15:49:25</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions and Pages.</p>
]]>
</content:encoded>
<pubDate>Mon, 19 Oct 2026 15:49:25 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmvffpp2p1hbw0ks8ecgqmnkr</link>
<guid>https://status.ubicloud.com/incident/cmvffpp2p1hbw0ks8ecgqmnkr</guid>
</item>
<item>
<title>Console is back up</title>
<description>
Type: Incident
Affected Components: Console
Oct 13, 22:24:23 GMT+0 - Resolved - Console is back up. This incident was automatically resolved by Instatus monitoring. Oct 13, 22:22:50 GMT+0 - Investigating - Console is down at the moment. This incident was automatically created by Instatus monitoring.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>22:24:23</var> GMT+0</small><br><strong>Resolved</strong> -
Console is back up. This incident was automatically resolved by Instatus monitoring..</p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>22:22:50</var> GMT+0</small><br><strong>Investigating</strong> -
Console is down at the moment. This incident was automatically created by Instatus monitoring..</p>
]]>
</content:encoded>
<pubDate>Tue, 13 Oct 2026 22:22:50 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmv78yh0i00io07qj3zeeubuw</link>
<guid>https://status.ubicloud.com/incident/cmv78yh0i00io07qj3zeeubuw</guid>
</item>
<item>
<title>Increased provisioning times for GitHub runners</title>
<description>
Type: Incident
Duration: 4 hours and 32 minutes
Affected Components: Ubicloud Runners
Oct 19, 14:12:07 GMT+0 - Postmortem - # Summary
On October 13, 2026, from 16:24 to 19:50 UTC, GitHub Actions jobs on Ubicloud runners took much longer than usual to start. Lock contention in our control-plane database cut runner placement to about half its normal rate, and the backlog was cleared by 21:10 UTC.
# Impact
* **Job start times:** between 16:20 and 21:00 UTC, jobs waited a median of about 4 minutes for a runner (normally 14 seconds), and 1 in 20 waited more than 28 minutes.
* **Throughput:** from 16:20 to 20:00 UTC we started about half as many runners as on a normal day.
# What happened
Our control plane tracks each host&#039;s free capacity in one database row per host, and every runner placement locks and then updates the row of the host it picks. On October 13, the runner fleet had high utilization, so concurrent placements often picked the same hosts. At the same time, a sudden burst of additional requested runners triggered an edge case in our allocation algorithm. This edge case more than tripled the rate of capacity checks.
At 16:24 UTC the database tipped into a slow state, most likely in PostgreSQL&#039;s row-locking bookkeeping (MultiXacts). Scans of the capacity table slowed from 0.1s to 2s, and slower placements overlapped even more on the same rows, which made the contention worse. The system stayed in that state until we cleared the backlog of waiting runners.
Our admission checks made it worse. They are not atomic: runners that check at the same moment all see the same free capacity and are let through together. Each time capacity seemed to free up, thousands of runners entered placement at once and competed for the little space left, which twice brought the slowdown back after we had cleared it.
We apologize for the impact this event caused our customers.
# Timeline (UTC, October 13)
| **Time (UTC)** | **Event** |
| -------------- | -------------------------------------------------------------------------------------------------------------------- |
| 16:24 | Runner placement slows sharply and job start times begin to climb. |
| 16:37 | We begin mitigating. Adding workers and rolling back a recent deployment do not help. |
| 17:56 | Reducing worker concurrency eases the database lock contention. |
| 18:07 - 19:50 | We clear the queue of runners waiting for placement. After two brief relapses, placement recovers for good at 19:50. |
| 21:10 | The backlog is fully drained and job start times are back to normal. |
# What we&#039;re changing
**Already merged**
* Placements now lock a host before using it and skip hosts that another placement holds, so two placements no longer work on the same database row at once (merged October 19,[ #6680](https://github.com/ubicloud/ubicloud/pull/6680)). In a lab replay of the incident&#039;s load, this change removed the slowdown.
* IP addresses and CPUs are reserved without waiting on other placements (October 13-14,[ #6671](https://github.com/ubicloud/ubicloud/pull/6671),[ #6683](https://github.com/ubicloud/ubicloud/pull/6683)).
* Runners created to catch up on queued jobs now start over a short window instead of all at the same moment (October 16,[ #6681](https://github.com/ubicloud/ubicloud/pull/6681)).
**In progress**
* Admission will reserve capacity atomically, so a burst can only start as many runners as there is room for. Waiting runners will share one capacity check and be admitted gradually.
* Created a simulator to simulate our runner allocation algorithm to ensure that we don’t run into similar edge cases. Once we confirm the simulator reflects the production environment’s behavior, we will make further updates to our allocation and scheduling logic. Oct 13, 21:28:30 GMT+0 - Resolved - This incident has been resolved. We are going to prepare and provide a retrospective in the upcoming days. Oct 13, 21:15:15 GMT+0 - Monitoring - The fix that we implemented didn&#039;t have the intended effect due to a high backlog. We took manual action to reduce the backlog. Queue times should be back to normal now. We&#039;re monitoring the situation. Oct 13, 18:22:47 GMT+0 - Identified - We identified the issue&#039;s root cause and we implemented a fix. We expect the fix to start improving the queue times in a short amount of time. Oct 13, 16:56:44 GMT+0 - Investigating - Our control plane has been experiencing increased provisioning times for GitHub runners. This may result in longer wait times for jobs to start. We are actively investigating the issue and will provide updates as we have more information.
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 4 hours and 32 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 19</var>, <var data-var='time'>14:12:07</var> GMT+0</small><br><strong>Postmortem</strong> -
# Summary
On October 13, 2026, from 16:24 to 19:50 UTC, GitHub Actions jobs on Ubicloud runners took much longer than usual to start. Lock contention in our control-plane database cut runner placement to about half its normal rate, and the backlog was cleared by 21:10 UTC.
# Impact
* **Job start times:** between 16:20 and 21:00 UTC, jobs waited a median of about 4 minutes for a runner (normally 14 seconds), and 1 in 20 waited more than 28 minutes.
* **Throughput:** from 16:20 to 20:00 UTC we started about half as many runners as on a normal day.
# What happened
Our control plane tracks each host&#039;s free capacity in one database row per host, and every runner placement locks and then updates the row of the host it picks. On October 13, the runner fleet had high utilization, so concurrent placements often picked the same hosts. At the same time, a sudden burst of additional requested runners triggered an edge case in our allocation algorithm. This edge case more than tripled the rate of capacity checks.
At 16:24 UTC the database tipped into a slow state, most likely in PostgreSQL&#039;s row-locking bookkeeping (MultiXacts). Scans of the capacity table slowed from 0.1s to 2s, and slower placements overlapped even more on the same rows, which made the contention worse. The system stayed in that state until we cleared the backlog of waiting runners.
Our admission checks made it worse. They are not atomic: runners that check at the same moment all see the same free capacity and are let through together. Each time capacity seemed to free up, thousands of runners entered placement at once and competed for the little space left, which twice brought the slowdown back after we had cleared it.
We apologize for the impact this event caused our customers.
# Timeline (UTC, October 13)
| **Time (UTC)** | **Event** |
| -------------- | -------------------------------------------------------------------------------------------------------------------- |
| 16:24 | Runner placement slows sharply and job start times begin to climb. |
| 16:37 | We begin mitigating. Adding workers and rolling back a recent deployment do not help. |
| 17:56 | Reducing worker concurrency eases the database lock contention. |
| 18:07 - 19:50 | We clear the queue of runners waiting for placement. After two brief relapses, placement recovers for good at 19:50. |
| 21:10 | The backlog is fully drained and job start times are back to normal. |
# What we&#039;re changing
**Already merged**
* Placements now lock a host before using it and skip hosts that another placement holds, so two placements no longer work on the same database row at once (merged October 19,[ #6680](https://github.com/ubicloud/ubicloud/pull/6680)). In a lab replay of the incident&#039;s load, this change removed the slowdown.
* IP addresses and CPUs are reserved without waiting on other placements (October 13-14,[ #6671](https://github.com/ubicloud/ubicloud/pull/6671),[ #6683](https://github.com/ubicloud/ubicloud/pull/6683)).
* Runners created to catch up on queued jobs now start over a short window instead of all at the same moment (October 16,[ #6681](https://github.com/ubicloud/ubicloud/pull/6681)).
**In progress**
* Admission will reserve capacity atomically, so a burst can only start as many runners as there is room for. Waiting runners will share one capacity check and be admitted gradually.
* Created a simulator to simulate our runner allocation algorithm to ensure that we don’t run into similar edge cases. Once we confirm the simulator reflects the production environment’s behavior, we will make further updates to our allocation and scheduling logic..</p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>21:28:30</var> GMT+0</small><br><strong>Resolved</strong> -
This incident has been resolved. We are going to prepare and provide a retrospective in the upcoming days..</p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>21:15:15</var> GMT+0</small><br><strong>Monitoring</strong> -
The fix that we implemented didn&#039;t have the intended effect due to a high backlog. We took manual action to reduce the backlog. Queue times should be back to normal now. We&#039;re monitoring the situation..</p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>18:22:47</var> GMT+0</small><br><strong>Identified</strong> -
We identified the issue&#039;s root cause and we implemented a fix. We expect the fix to start improving the queue times in a short amount of time..</p>
<p><small>Oct <var data-var='date'> 13</var>, <var data-var='time'>16:56:44</var> GMT+0</small><br><strong>Investigating</strong> -
Our control plane has been experiencing increased provisioning times for GitHub runners. This may result in longer wait times for jobs to start. We are actively investigating the issue and will provide updates as we have more information..</p>
]]>
</content:encoded>
<pubDate>Tue, 13 Oct 2026 16:56:44 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmv6xb45g01kp1bpk93r1naqy</link>
<guid>https://status.ubicloud.com/incident/cmv6xb45g01kp1bpk93r1naqy</guid>
</item>
<item>
<title>Incident with Actions, Pull Requests and Issues</title>
<description>
Type: Incident
Duration: 19 days, 18 hours and 19 minutes
Affected Components: GitHub → Actions
Oct 12, 20:16:07 GMT+0 - Resolved - On October 12th 2026 11:51 UTC GitHub experienced elevated 500 responses across several application pages. This caused failures on Issues and Pull Requests as well as Projects and Actions. Customers also experienced delayed notifications in their web inbox and via email. &lt;br /&gt;&lt;br /&gt;The immediate impact to Issues, Pull Requests, Projects, and Actions was mitigated by 12:41 UTC on October 12th. Notifications deliveries to the web inbox and via email were recovering at 15:27 UTC and fully mitigated by 20:16 UTC. &lt;br /&gt;&lt;br /&gt;The incident was triggered through a latent condition in our notifications delivery pipeline that put a high volume of an expensive query on our sharded database replicas. This reduced available database capacity to serve requests for Issues, Pull Requests, Projects, and Actions.&lt;br /&gt;&lt;br /&gt;We have fixed the notifications pipeline’s code path that issued the expensive query. We are also improving resilience of the database layer to reduce the likelihood and impact of similar incidents. Oct 12, 19:28:42 GMT+0 - Investigating - Web and email notifications will be delayed for 1 more hour as we are close to completing the backlog of notifications. Oct 12, 17:34:24 GMT+0 - Investigating - Web and email notifications continue to be delayed as we work through a backlog of notifications. Oct 12, 15:50:39 GMT+0 - Investigating - Notifications have resumed but will continue to be delayed as we work through a backlog. Oct 12, 15:19:00 GMT+0 - Investigating - We have successfully applied a mitigation for notifications, and notifications have resumed but will continue to be delayed as we work through a backlog. Oct 12, 14:37:49 GMT+0 - Investigating - We have mitigated the issues with Actions, issues, and projects, and have a mitigation in progress to address the delay with notifications. Oct 12, 14:36:09 GMT+0 - Investigating - The degradation affecting Actions has been mitigated. We are monitoring to ensure stability. Oct 12, 14:35:49 GMT+0 - Investigating - The degradation affecting Issues has been mitigated. We are monitoring to ensure stability. Oct 12, 13:38:44 GMT+0 - Investigating - Our initial mitigation was not fully successful, and users may still experience delays and errors with Actions, issues and projects, and notifications are delayed. We&#039;re continuing to investigate the root cause and work on additional mitigations. Oct 12, 13:02:17 GMT+0 - Investigating - Pull Requests is operating normally. Oct 12, 12:57:44 GMT+0 - Investigating - We have a mitigation in progress and are monitoring for recovery. Users may also experience delayed web notifications as we mitigate. Oct 12, 12:33:21 GMT+0 - Investigating - Users may experience issues loading the Actions tab UI, interacting with the Checks API, queueing new Actions runs, creating and updating issues, and requesting PR reviews, including via REST and GraphQL APIs. Oct 12, 12:21:48 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions, Issues and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 19 days, 18 hours and 19 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>20:16:07</var> GMT+0</small><br><strong>Resolved</strong> -
On October 12th 2026 11:51 UTC GitHub experienced elevated 500 responses across several application pages. This caused failures on Issues and Pull Requests as well as Projects and Actions. Customers also experienced delayed notifications in their web inbox and via email. &lt;br /&gt;&lt;br /&gt;The immediate impact to Issues, Pull Requests, Projects, and Actions was mitigated by 12:41 UTC on October 12th. Notifications deliveries to the web inbox and via email were recovering at 15:27 UTC and fully mitigated by 20:16 UTC. &lt;br /&gt;&lt;br /&gt;The incident was triggered through a latent condition in our notifications delivery pipeline that put a high volume of an expensive query on our sharded database replicas. This reduced available database capacity to serve requests for Issues, Pull Requests, Projects, and Actions.&lt;br /&gt;&lt;br /&gt;We have fixed the notifications pipeline’s code path that issued the expensive query. We are also improving resilience of the database layer to reduce the likelihood and impact of similar incidents..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>19:28:42</var> GMT+0</small><br><strong>Investigating</strong> -
Web and email notifications will be delayed for 1 more hour as we are close to completing the backlog of notifications..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>17:34:24</var> GMT+0</small><br><strong>Investigating</strong> -
Web and email notifications continue to be delayed as we work through a backlog of notifications..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>15:50:39</var> GMT+0</small><br><strong>Investigating</strong> -
Notifications have resumed but will continue to be delayed as we work through a backlog..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>15:19:00</var> GMT+0</small><br><strong>Investigating</strong> -
We have successfully applied a mitigation for notifications, and notifications have resumed but will continue to be delayed as we work through a backlog..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>14:37:49</var> GMT+0</small><br><strong>Investigating</strong> -
We have mitigated the issues with Actions, issues, and projects, and have a mitigation in progress to address the delay with notifications..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>14:36:09</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting Actions has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>14:35:49</var> GMT+0</small><br><strong>Investigating</strong> -
The degradation affecting Issues has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>13:38:44</var> GMT+0</small><br><strong>Investigating</strong> -
Our initial mitigation was not fully successful, and users may still experience delays and errors with Actions, issues and projects, and notifications are delayed. We&#039;re continuing to investigate the root cause and work on additional mitigations..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>13:02:17</var> GMT+0</small><br><strong>Investigating</strong> -
Pull Requests is operating normally..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>12:57:44</var> GMT+0</small><br><strong>Investigating</strong> -
We have a mitigation in progress and are monitoring for recovery. Users may also experience delayed web notifications as we mitigate..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>12:33:21</var> GMT+0</small><br><strong>Investigating</strong> -
Users may experience issues loading the Actions tab UI, interacting with the Checks API, queueing new Actions runs, creating and updating issues, and requesting PR reviews, including via REST and GraphQL APIs..</p>
<p><small>Oct <var data-var='date'> 12</var>, <var data-var='time'>12:21:48</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions, Issues and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Mon, 12 Oct 2026 12:21:47 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmv584qaz2ajm0rp4khp7n9do</link>
<guid>https://status.ubicloud.com/incident/cmv584qaz2ajm0rp4khp7n9do</guid>
</item>
<item>
<title>Incident with Git Operations, Issues, Actions and Pull Requests</title>
<description>
Type: Incident
Duration: 1 day, 23 hours and 43 minutes
Affected Components: GitHub → Actions
Oct 7, 18:04:49 GMT+0 - Resolved - On October 7, 2026, from 16:52 to 17:01 UTC, GitHub experienced a recurrence of the write-operation outage from earlier that day. Both outages shared the same root cause. Please refer to the combined summary covering both incidents, their impact, and our prevention measures: https://www.githubstatus.com/incidents/djlmxz2zd0j7 Oct 7, 18:00:49 GMT+0 - Monitoring - All systems have recovered. We have applied mitigations and do not anticipate another recurrence. Oct 7, 17:27:12 GMT+0 - Monitoring - The degradation has been mitigated. We are monitoring to ensure stability. Oct 7, 17:25:26 GMT+0 - Investigating - We observed widespread impact across GitHub services from 16:52 UTC to 17:01 UTC, and are investigating the root cause. All systems have recovered. This incident is a reoccurrence of the incident from earlier in the day https://www.githubstatus.com/incidents/djlmxz2zd0j7. Durable mitigation is in progress. We will provide another update in 30 minutes. Oct 7, 17:17:05 GMT+0 - Investigating - We are investigating reports of degraded performance for Actions, Git Operations, Issues and Pull Requests
</description>
<content:encoded>
<![CDATA[<p><strong>Type:</strong> Incident</p>
<p><strong>Duration:</strong> 1 day, 23 hours and 43 minutes</p>
<p><strong>Affected Components:</strong> </p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>18:04:49</var> GMT+0</small><br><strong>Resolved</strong> -
On October 7, 2026, from 16:52 to 17:01 UTC, GitHub experienced a recurrence of the write-operation outage from earlier that day. Both outages shared the same root cause. Please refer to the combined summary covering both incidents, their impact, and our prevention measures: https://www.githubstatus.com/incidents/djlmxz2zd0j7.</p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>18:00:49</var> GMT+0</small><br><strong>Monitoring</strong> -
All systems have recovered. We have applied mitigations and do not anticipate another recurrence..</p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>17:27:12</var> GMT+0</small><br><strong>Monitoring</strong> -
The degradation has been mitigated. We are monitoring to ensure stability..</p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>17:25:26</var> GMT+0</small><br><strong>Investigating</strong> -
We observed widespread impact across GitHub services from 16:52 UTC to 17:01 UTC, and are investigating the root cause. All systems have recovered. This incident is a reoccurrence of the incident from earlier in the day https://www.githubstatus.com/incidents/djlmxz2zd0j7. Durable mitigation is in progress. We will provide another update in 30 minutes..</p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>17:17:05</var> GMT+0</small><br><strong>Investigating</strong> -
We are investigating reports of degraded performance for Actions, Git Operations, Issues and Pull Requests.</p>
]]>
</content:encoded>
<pubDate>Wed, 7 Oct 2026 17:17:05 +0000</pubDate>
<link>https://status.ubicloud.com/incident/cmuydgr3804cn0spn8t5tpv2w</link>
<guid>https://status.ubicloud.com/incident/cmuydgr3804cn0spn8t5tpv2w</guid>
</item>
</channel>
</rss>