On September 15th at approximately 8:40 am MDT, NetDocuments experienced a disruption with the US Service.   During the event customers would have experienced slowness in downloading and opening documents or in some instances would have received an error from ndOffice when attempting to perform this action.

We were initially alerted to the start of the issue via our automated monitoring infrastructure and were able to begin investigating immediately.  By approximately 3:09 pm MDT the Service had been stabilized and download speeds had returned to more normal levels.  Additional performance tuning continued until 6:00 pm MDT.

An investigation found the root cause was related to a change in the recently deployed 23.18 version of the Service dealing with the method used to connect to an analytics logging function.  The change introduced delays in the logging call that affected the download transactions.  This resulted in the slowness experienced for customers on the US Service.   Once the problem was fully identified the Service was rolled back to the previously stable 23.17 version and performance tuning was implemented on the logging function.  An updated fix has since been implemented in 23.18 and the new version has been redeployed.

As a part of ongoing and continuous improvement, NetDocuments is evaluating and adding additional test cases to enhance the Beta and Deployment processes to prevent a similar defect from being able to affect the Service in this way again.

We apologize for any inconvenience this disruption may have caused.

Thank you,

The NetDocuments Team