Over the last fortnight the operation of the telescope has been somewhat disrupted. This reduction in the capacity of the service has been a result of two unconnected reasons.
Dome code development
The first of the reasons is development of new code to control the dome. Over the last six months we have have made various improvements to the code controlling the dome in an attempt to minimise its impact on images. The dome now attempts to track the mount as it moves in real time through the imaging process. We are hoping to add new code soon which will make the dome aware of the extent to which the dome is obscuring the imaging camera, this will allow the dome to make more intelligent decisions about when it is necessary to rotate to follow the camera.
The changes to the dome code have resulted in a large increase in the amount of communication with the dome, and also have changed the communication from happening in a single thread, to requests coming in from multiple threads. The existing dome control code is thread safe, however the increase in communication appears to now be overwhelming the dome controller; this problem has necessitated a change in the way in which we communicate with the dome. The alterations to the dome control code have required numerous changes throughout the system. Testing and refining of this code has required frequent restarts of the controlserver software. In some circumstances the restart of the controlserver software resets the cool count of the system to zero, this effectively takes the Galaxy camera off-line as it waits to cool again. Whilst this is frustrating, the system does continue to image on the Constellation and Cluster cameras, minimising the amount of lost observing time. I have talked with Chris and believe that we now have a way to indicate on the website when new code development is happening.
Mount pointing errors
We have recently started to have pointing errors with the mount. We are still working to understand the cause of these errors. The system is designed to reject images which have a pointing error in excess of certain predetermined values. Whilst we continue to work on this problem a larger than normal number of the images taken on Galaxy camera will be rejected and discarded by the system. These rejected images are shown in the RTD feed, but their imaging is not recorded in the list of work undertaken in the last night's jobs page; this often makes it looks like the system hasn't done much work in a night.
We have not reached the end of the dome code development cycle, or resolved the mount pointing errors yet. As we move forward with the work we hope to keep you all a bit better informed about what is going on.