CHPC - Degraded file system performance with various impacts (Open OnDemand sessions not starting, reduced input and output performance) – Incident details

Degraded file system performance with various impacts (Open OnDemand sessions not starting, reduced input and output performance)

ResolvedDegraded performance20 hours 35 minutes

Affected

General Environment (GE)

Degraded performance from 7:30 PM to 1:50 AM, Operational from 1:50 AM to 4:06 PM

Open OnDemand

Degraded performance from 7:30 PM to 1:50 AM, Operational from 1:50 AM to 4:06 PM

  • ResolvedUTC

    This incident has been resolved. CHPC systems have returned to responsive states. If you continue to encounter issues, please contact the CHPC at helpdesk@chpc.utah.edu.

  • MonitoringUTC

    CHPC staff have identified a number of jobs that were impacting file system performance. The team at the CHPC canceled jobs with a significant impact, which has improved performance. Systems, including Open OnDemand, should now be more responsive. CHPC staff will continue to monitor the situation.

  • InvestigatingUTC

    We are receiving user reports of Open OnDemand sessions not starting. So far, issues have been reported on notchpeak and kingspeak but appear to be intermittent. A retry may resolve the issue in some cases. The CHPC is currently investigating the cause.