[SCHEDULER-1797]  Prevent ugly error if ACP disappears
Type Bug
Priority High
Severity Minor
Component Dispatch Scheduler Engine
Fixed In Version [9.0] 9.0
Versions Affected [8.3] 8.3
Severity Closed
Resolution Complete
Reported By Bob Denny
Resources Bob Denny
Start Date 1/12/2021

Description 
Somehow, catch the various ways in which a dead ACP will cause a last-chance error to come up. Log an error an error like






Scheduler lost contact with ACP, probably as a result of ACP getting locked up from a device communications failure. This is almost certainly not a software bug.







and then go to Operator Intervention.

Comments
1/25/2022 11:09:11 AM   Bob Denny 

 

  SVN Comment
 

 

  Author
  rbdenny
 

 

  Repository
  svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
 

 

  SVN Revision
  909
 

 

  Affected files
  /trunk/Scheduler/ACPSequencer.cs (Modified)

 

 

  Check-in comment
  More bulletproofing against ACP loss.
GEM:1797
1/25/2022 10:09:39 AM   Bob Denny 
Well after making the problem "clean" in ACP (ACP-1875) now Scheduler has problems trying ti exit after getting to Operator Intervention.
1/24/2022 2:43:39 PM   Bob Denny 

 

  SVN Comment
 

 

  Author
  rbdenny
 

 

  Repository
  svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
 

 

  SVN Revision
  908
 

 

  Affected files
  /trunk/Scheduler/AssemblyInfo.cs (Modified)
/trunk/Scheduler/Engine.cs (Modified)

 

 

  Check-in comment
  Catch ACP compromise caused by the exit of a connected device's LocalServer. Up Scheduler.exe to 9.0.3
GEM:1797
1/24/2022 2:38:05 PM   Bob Denny 
Catch ACP failure caused by the loss of a device driver, and detect that ACP is compromised. Not able to Alarm GNS through ACP.  GNS will eventually time out on this one because ACP will stop heartbeats.
1/17/2022 4:16:10 PM   Bob Denny 
BUG






See this Comm Center thread by Nicklas Storck. His 10 Micron locks up, locking up ACP, and when the Scheduler decides to Alarm GNS going to Operator intervention, it gets to the damn last-chance exception handler. Look at the error traceback. 
10/27/2021 2:33:57 PM   Bob Denny 
Yeah, well there was more work in a lot of places. Caught during final testing.
10/27/2021 2:32:44 PM   Bob Denny 

 

  SVN Comment
 

 

  Author
  rbdenny
 

 

  Repository
  svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
 

 

  SVN Revision
  895
 

 

  Affected files
  /trunk/Database/Database.cs (Modified)
/trunk/Scheduler/ACPSequencer.cs (Modified)
/trunk/Scheduler/Engine.cs (Modified)
/trunk/Scheduler/Logger.cs (Modified)
/trunk/Scheduler/MainForm.cs (Modified)

 

 

  Check-in comment
  Miscellaneous changes to allow clean entry into Operator Intervention when there is a total failure of ACP.
GEM:1797
1/12/2021 2:09:12 PM   Bob Denny 

 

  SVN Comment
 

 

  Author
  rbdenny
 

 

  Repository
  svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
 

 

  SVN Revision
  846
 

 

  Affected files
  /trunk/Help/RelNotes.htm (Modified)
/trunk/Scheduler/AssemblyInfo.cs (Modified)
/trunk/Scheduler/Engine.cs (Modified)

 

 

  Check-in comment
  Make errors caused by ACP becoming compromised much more friendly and rational.
GEM:1797
1/12/2021 2:07:21 PM   Bob Denny 
This turned out to be easier than I thought. "Only" 6+ hours. I thought it would require a lot more armor plating. The main wrinkle was handling the unconverted COMExceptions on this low level failure of the inter-application LPC communication,
11/19/2020 10:10:24 AM   Bob Denny 
Here's a typical one that came today from Bruce McMath.