tag:status.yellowdig.com,2005:/historyYellowdig Status - Incident History2024-03-28T14:40:41-04:00Yellowdigtag:status.yellowdig.com,2005:Incident/201385732024-03-01T17:30:00-05:002024-03-01T17:57:43-05:00Page Not Found Errors<p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>17:30</var> EST</small><br><strong>Resolved</strong> - For a brief period, when non-members attempted to view a community they would see an incorrect "Page Not Found" display. This is is now resolved</p>tag:status.yellowdig.com,2005:Incident/197699732024-01-19T09:30:00-05:002024-01-19T16:35:12-05:00Video upload and recording issues<p><small>Jan <var data-var='date'>19</var>, <var data-var='time'>09:30</var> EST</small><br><strong>Resolved</strong> - A software bug has caused video file attachments and in-browser video recording to fail. Our team has identified the issue and is releasing a patch as soon as possible</p>tag:status.yellowdig.com,2005:Incident/192525842023-12-28T13:01:16-05:002023-12-28T13:01:16-05:00Important Maintenance Notice<p><small>Dec <var data-var='date'>28</var>, <var data-var='time'>13:01</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>28</var>, <var data-var='time'>10:30</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>15:04</var> EST</small><br><strong>Scheduled</strong> - Dear Yellowdig Community,<br /><br />As part of our ongoing efforts to enhance your experience with our platform, we are scheduling a maintenance window on December 28th, during which time the Yellowdig platform will be unavailable. Please take note of the following details:<br /><br />🗓️ Scheduled Maintenance Time: December 28th, 10:30 AM - 1:00 PM EST (3:30 PM - 6:00 PM UTC)<br /><br />🔨 Why Maintenance is Required: This maintenance is essential to ensure the optimal performance and reliability of our platform. We expect the actual platform downtime to be minimal, but we've allocated an extended maintenance window to ensure everything goes smoothly.<br /><br />📢 Stay Informed: We will keep you updated on the progress of the maintenance via our status page (https://status.yellowdig.com) and will notify you promptly once the work is complete.<br /><br />If you have any questions or concerns regarding this maintenance, please don't hesitate to reach out to our support team at support@yellowdig.com<br /><br />Thank you for your cooperation and for being a valued part of the Yellowdig community.<br /><br />Sincerely,<br />Team Yellowdig</p>tag:status.yellowdig.com,2005:Incident/183109972023-08-29T15:51:55-04:002023-08-29T15:51:55-04:00Issues with feed display<p><small>Aug <var data-var='date'>29</var>, <var data-var='time'>15:51</var> EDT</small><br><strong>Resolved</strong> - This issues has been resolved</p><p><small>Aug <var data-var='date'>29</var>, <var data-var='time'>15:14</var> EDT</small><br><strong>Identified</strong> - There's an issue with community feeds only displaying a few posts. The issue has been identified and a patch is in progress</p>tag:status.yellowdig.com,2005:Incident/182987102023-08-28T17:58:31-04:002023-08-28T17:58:31-04:00Issues with creating comments<p><small>Aug <var data-var='date'>28</var>, <var data-var='time'>17:58</var> EDT</small><br><strong>Resolved</strong> - The issue has been resolved</p><p><small>Aug <var data-var='date'>28</var>, <var data-var='time'>17:23</var> EDT</small><br><strong>Monitoring</strong> - Some code released today caused issues with creating comments. Some users may see "something went wrong" error messaging when opening the comment editor. The issue has been identified and a patch is being released shortly</p>tag:status.yellowdig.com,2005:Incident/175475082023-06-12T15:50:36-04:002023-06-12T15:50:36-04:00Issues with email sending<p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>15:50</var> EDT</small><br><strong>Resolved</strong> - Email sending services have now been restored. Search results still remain latent but will be drained and fully restored within the next few hours</p><p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>09:20</var> EDT</small><br><strong>Investigating</strong> - There's a Yellowdig wide email sending issue currently affecting the platform. Some invitation or notification emails may not be delivered. Investigation has started</p>tag:status.yellowdig.com,2005:Incident/173540542023-05-23T16:00:00-04:002023-05-24T12:07:10-04:00Loading error for communities with a post pinned to the top<p><small>May <var data-var='date'>23</var>, <var data-var='time'>16:00</var> EDT</small><br><strong>Resolved</strong> - At about 4:00pm eastern time a new code release caused communities using a post pinned to the top to start displaying an "error fetching data" message when loaded. This would have temporarily prevented members from accessing and participating in those communities. We were able to identify the source of the error and deploy a fix that restored normal functionality for all communities by 5:40pm. We apologize for any inconvenience this may have caused.</p>tag:status.yellowdig.com,2005:Incident/171843542023-05-08T15:37:55-04:002023-05-08T15:37:55-04:00Page loading error<p><small>May <var data-var='date'> 8</var>, <var data-var='time'>15:37</var> EDT</small><br><strong>Resolved</strong> - This is has been resolved</p><p><small>May <var data-var='date'> 8</var>, <var data-var='time'>15:34</var> EDT</small><br><strong>Identified</strong> - Yellowdig Engage experienced a momentary global outage due to a build pipeline failure. A fix is in progress. This issue affects the web application UI, APIs were not impacted</p>tag:status.yellowdig.com,2005:Incident/169965402023-04-28T18:18:06-04:002023-04-28T18:18:06-04:00Errors when posting in communities with freeform points enabled<p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>18:18</var> EDT</small><br><strong>Resolved</strong> - This issue has been resolved</p><p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>15:47</var> EDT</small><br><strong>Monitoring</strong> - A fix has been released to eliminate the error, a follow-up is in progress</p><p><small>Apr <var data-var='date'>28</var>, <var data-var='time'>14:52</var> EDT</small><br><strong>Identified</strong> - We've identified an issue where "Error fetching data" is shown when attempting to post in a community that has the Freeform points mode enabled. This issue was introduced by a change released earlier today. A patch is in progress</p>tag:status.yellowdig.com,2005:Incident/158742242023-01-18T13:00:35-05:002023-01-18T13:00:35-05:00System Instability<p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>13:00</var> EST</small><br><strong>Resolved</strong> - This is now resolved</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>11:38</var> EST</small><br><strong>Identified</strong> - A change released today caused some errors in production. The YD team is currently rolling back the release</p>tag:status.yellowdig.com,2005:Incident/158059022023-01-09T00:00:00-05:002023-01-09T17:38:04-05:00Increased Application Latency<p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>00:00</var> EST</small><br><strong>Resolved</strong> - Yellowdig Engage experienced an increase in page loading latency due to an increase in semester related traffic coupled with recent code changes introduced in late December. Application servers became overloaded and caused a cascading failure. Changes have been introduced to stabilize the issue, the Yellowdig team will continue to monitor</p>tag:status.yellowdig.com,2005:Incident/153527682022-12-16T15:01:31-05:002022-12-16T15:01:31-05:00Deployment Configuration Issue<p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>15:01</var> EST</small><br><strong>Resolved</strong> - This configuration issue is now resolved</p><p><small>Dec <var data-var='date'>16</var>, <var data-var='time'>14:37</var> EST</small><br><strong>Identified</strong> - Yellowdig shipped an incorrect configuration change which resulted in site-wide login and access issues. The root cause has been identified and a rollback is in progress</p>tag:status.yellowdig.com,2005:Incident/112302542022-09-23T16:25:32-04:002022-09-23T16:25:32-04:00Write operations failing<p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>16:25</var> EDT</small><br><strong>Resolved</strong> - All systems are now stabilized, postmortem review ongoing, we appreciate everyone's patience with these system issues and are actively discussing long term process improvements</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>09:57</var> EDT</small><br><strong>Monitoring</strong> - The long running task has completed, transaction workloads are back to normal, we're continuing to monitor</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>04:53</var> EDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>02:13</var> EDT</small><br><strong>Identified</strong> - Yellowdig engage is experiencing a widespread outage related to write operations including LTI launches and post submissions. Investigation ongoing</p>tag:status.yellowdig.com,2005:Incident/112291432022-09-23T00:31:25-04:002022-09-23T00:31:25-04:00Issues with LTI Launching and Post Submission<p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>00:31</var> EDT</small><br><strong>Resolved</strong> - Yellowdig Engage users experienced a high rate of transaction failures between 12:03 and 12:23 AM EDT. The team has been notified and corrective measures are being applied</p>tag:status.yellowdig.com,2005:Incident/111962822022-09-20T14:58:58-04:002022-09-20T14:58:58-04:00Yellowdig Engage Platform Instability<p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>14:58</var> EDT</small><br><strong>Resolved</strong> - The system has stabilized, we're now conducting a postmortem</p><p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>14:23</var> EDT</small><br><strong>Monitoring</strong> - A recent release caused increased latency and page loading timeouts. An investigation in progress</p>tag:status.yellowdig.com,2005:Incident/109013462022-08-23T13:00:00-04:002022-08-24T11:50:52-04:00Notification Sending Issues<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>13:00</var> EDT</small><br><strong>Resolved</strong> - An infrastructure change made at 1PM EDT 8/23 impacted the sending of in-app and email notifications. Yellowdig users did not receive notifications for posts created during the affected period. The issue was patched the following morning (8/24) at 8am. The Yellowdig development team is beginning investigation into making the notification pipeline more durable. No other Yellowdig functionality was impacted by this change</p>tag:status.yellowdig.com,2005:Incident/107382372022-08-01T15:29:27-04:002022-08-01T15:29:27-04:00Post Rendering Issues<p><small>Aug <var data-var='date'> 1</var>, <var data-var='time'>15:29</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'> 1</var>, <var data-var='time'>15:20</var> EDT</small><br><strong>Monitoring</strong> - A release today introduced a site-wide post rendering error. Users may have seen "Sorry, this component can't be shown because of an error" when attempting to view the Yellowdig community page. A rollback is in progress. Other Yellowdig features were not impacted by this change</p>tag:status.yellowdig.com,2005:Incident/106290642022-07-18T16:30:31-04:002022-07-18T16:30:31-04:00503 Service Temporarily Unavailable Errors<p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>16:30</var> EDT</small><br><strong>Resolved</strong> - The release has been rolled back</p><p><small>Jul <var data-var='date'>18</var>, <var data-var='time'>16:18</var> EDT</small><br><strong>Identified</strong> - A deployment released today (July 18th) caused Yellowdig users to see intermittent 503 error pages. The Yellowdig team has identified the issue and is rolling back the release</p>tag:status.yellowdig.com,2005:Incident/103957372022-06-23T06:30:00-04:002022-06-24T10:19:48-04:00Incorrect point calculation for freeform points mode, Page loading errors<p><small>Jun <var data-var='date'>23</var>, <var data-var='time'>06:30</var> EDT</small><br><strong>Resolved</strong> - An application bug was released 6/23 which caused incorrect point calculation for communities using the "freeform" points mode. When the release was rolled back, users experienced page loading errors until a manual operation was run to restore service</p>tag:status.yellowdig.com,2005:Incident/100630562022-05-24T06:44:39-04:002022-05-24T06:44:39-04:00Post Submission Issues<p><small>May <var data-var='date'>24</var>, <var data-var='time'>06:44</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>24</var>, <var data-var='time'>06:44</var> EDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>May <var data-var='date'>24</var>, <var data-var='time'>06:35</var> EDT</small><br><strong>Monitoring</strong> - Yellowdig has identified a browser issue that is preventing posts from being submitted. A patch release is in progress</p>tag:status.yellowdig.com,2005:Incident/100058832022-05-15T12:00:00-04:002022-05-16T16:48:54-04:00High incidence of "Oh Snap" errors<p><small>May <var data-var='date'>15</var>, <var data-var='time'>12:00</var> EDT</small><br><strong>Resolved</strong> - We experienced a period from about 12:00 noon until 2:30pm on Sunday, May 15th, where many actions in the platform would generate errors. This resulted in a seriously degraded experience for users during that time. We apologize for any inconvenience.</p>tag:status.yellowdig.com,2005:Incident/97833592022-04-14T11:59:24-04:002022-04-14T11:59:24-04:00Increased Application Latency<p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>11:59</var> EDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>11:46</var> EDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>11:45</var> EDT</small><br><strong>Monitoring</strong> - The cause has been identified and systems are now stable, we're planning to monitor over next few hours</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>11:44</var> EDT</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>11:17</var> EDT</small><br><strong>Investigating</strong> - Yellowdig Engage is experiencing an increase application latency, the team is investigating</p>tag:status.yellowdig.com,2005:Incident/95609752022-03-17T11:17:24-04:002022-03-17T11:17:24-04:00Degraded performance<p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>11:17</var> EDT</small><br><strong>Resolved</strong> - The site continues to be stable so we are considering this incident resolved. Sorry for any inconvenience this may have caused.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:41</var> EDT</small><br><strong>Monitoring</strong> - A fix has been implemented and appears to have fixed the latency and access issues as of 7:40 ET. We will continue monitoring the site to ensure everything is stable.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:33</var> EDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>18:40</var> EDT</small><br><strong>Investigating</strong> - We are currently investigating and taking action on an event that has caused increased latency of the platform and some users to be unable to LTI launch or login. Some users would have started to notice issues starting at around 6:40 ET.</p>tag:status.yellowdig.com,2005:Incident/93923732022-02-23T14:47:36-05:002022-02-23T14:47:36-05:00Access Issues<p><small>Feb <var data-var='date'>23</var>, <var data-var='time'>14:47</var> EST</small><br><strong>Resolved</strong> - Yellowdig Engage experienced a brief period service failure related to a configuration change</p>tag:status.yellowdig.com,2005:Incident/91499342022-01-24T02:57:22-05:002022-01-24T02:57:23-05:00Degraded Service (2)<p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>02:57</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>02:12</var> EST</small><br><strong>Investigating</strong> - Yellowdig Engage is experiencing degraded service and high latency. Our team is investigating the issue</p>