[SCHEDULER-1044]  Time Spacing for Linked Observations
Type Quirk
Priority Medium
Severity Minor
Component Whole Product
Fixed In Version [8.08.0
Versions Affected [3.5.33.5.3
Severity Closed
Resolution Complete
Reported By Bob Denny
Resources Bob Denny
Start Date 11/16/2014

Description
What is the meaning of a plus/minus on the time spacing? Why is there a minus at all. See this Comm Center thread by Fraser Clark.

Comments
12/9/2014 11:29:56 PM   Bob Denny
SVN Comment
Author rbdenny
Repository svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
SVN Revision 477
Affected files /trunk/Database/Database.cs (Modified)
/trunk/Empty SchedulerData.mdb (Modified)
/trunk/Help/RelNotes.htm (Modified)
/trunk/SchedulerSetup/Custom Actions/UpdateDatabase.vbs (Modified)
Check-in comment The dispatcher will now insert Plans between linked Observations of running Plans if the Plan to be inserted will fit before the nominal start time of the upcoming linked Observation instead of the earliest time. Thus longer running Plans can be fit into the gaps between linked Observations of running Plans. Mostly a revision to the SelectNominalTimesForRunningObservations query in the DB. GEM:1044
12/9/2014 11:05:45 PM   Bob Denny
The essence of this is a changed stored procedure/query SelectNominalTimesForRunningObservations:

SELECT Observation.Name AS ObservationName, Plan.Name AS PlanName, Observation.EarliestTime AS NominalStart, Observation.EarliestTime+(Observation.EstimatedTime/86400) AS NominalEnd
FROM Plan INNER JOIN Observation ON Plan.ID=Observation.PlanID
WHERE Plan.Status=2 And Observation.Status=0;

Becomes 

SELECT Observation.Name AS ObservationName, Plan.Name AS PlanName, Observation.EarliestTime+(Observation.TimingTolerance/86400) AS NominalStart, Observation.EarliestTime+((Observation.TimingTolerance+Observation.EstimatedTime)/86400) AS NominalEnd
FROM Plan INNER JOIN Observation ON Plan.ID = Observation.PlanID
WHERE (((Plan.Status)=2) AND ((Observation.Status)=0));

The effect is to allow Plans to be started will can run up to the Nominal start time of an upcoming linked Observation, instead of needing to fit before its Earliest time.
11/16/2014 6:34:30 PM   Bob Denny
SVN Comment
Author rbdenny
Repository svn+ssh://rbdenny@a2_svn_dc3/home/rbdenny/svn/astro/scheduler
SVN Revision 431
Affected files /trunk/White Papers/ACP Scheduler/Scheduler 2014.docx (Added)
Check-in comment New Scheduler White Paper which will reflect the change to the rules for the improved time slotting, as well as an eventual phasing system for observations of periodic variables like RR-Lyrae and Cepheids. GEM:1044
11/16/2014 6:07:17 PM   Bob Denny
Sent email to him, maybe we can talk.
11/16/2014 5:53:05 PM   Bob Denny
I'm a bit confused about the time spacing/time tolerance logic in linked observations (best effort). I can't find a detailed explanation of this in the manuals, or on the forums -- sorry if I've missed it!

My 'understanding' of this is that the next linked observation can be done within the time window defined by (??) time spacing +/- time tolerance. The scheduler will postpone new plans which would override a linked observation in an existing plan. However, it seems to take a very aggressive use of the 'time tolerance' parameter, and treat this rather like a "I absolutely must start the plan at the beginning of the window" parameter. That means it seems to block out a huge amount of the night, and ends up pushing the spacing of linked observations to the very minimum spacing (effectively "time spacing - time tolerance"), rather than the average spacing ("time spacing"). 

My example is a linked plan of observations (of the same target) with a time spacing of 5400s and a time tolerance of 1800s. I want a repeat observation roughly every 5400s, but I don't mind if it is within 1800s (+/-) of that. Each observation takes about 900s. I'd hoped that would give me enough freedom in the schedule to intersperse ~3600s observations of other targets throughout the night. However, what I find is that scheduler will come to the end of the first linked observation (at T+900s), and then veto any 3600s long plans (which would finish at T+4500s) as they overlap with the next slot for the linked observation (which starts at T+5400-1800 = T+3600s). Here's an extract from the log file;

15-Nov-2013 00:06:07.5: Send Observation NGC188 to ACP Sequencer
15-Nov-2013 00:06:17.5: Sequencer is now active
15-Nov-2013 00:20:03.9: Sequencer is no longer active
15-Nov-2013 00:20:03.9: Post-job status check done (stat=Completed)
15-Nov-2013 00:20:03.9: Acquisition time: 836.3647277 sec.
15-Nov-2013 00:20:03.9: Data for Observation NGC188 acquired successfully.
15-Nov-2013 00:20:04.3: Image Efficiency: 59.6%
15-Nov-2013 00:20:04.3: Cycle Efficiency: 99.8%
15-Nov-2013 00:20:04.6: SelectRunning: Selecting from 1 running plans:
15-Nov-2013 00:20:04.6: Plan NGC188-sparse next Obs is NGC188
15-Nov-2013 00:20:04.6: Not due for at least 3662 sec. (skip)
15-Nov-2013 00:20:04.7: Obs M76 in Plan M76-H-alpha (9-12) would overlap Obs NGC188 in running Plan NGC188-sparse -- vetoed


Now that M76-H-alpha plan runs for about 4000s. So, it won't fit in before the start of the next time slot for NGC188 (as Scheduler correctly calculates) -- but it would easily fit in before the end of the time slot. Now the scheduler just sits there dispatching increasingly short plans until nothing fits into the gap before the start of the next linked observation window. Then it sits idle until the next linked window opens up, whence it dispatches the next linked observation at the very earliest opportunity. 

That doesn't seem like the right meaning of 'time tolerance' -- as it's only ever working in one direction in scheduler. It seems to completely defeat the purpose of having a large time tolerance to give flexibility. I understand the logic for the time tolerance if a plan runs over it's expected time, but it seems it could do a lot more than that and give a lot more flexibility in scheduling. 

Am I misunderstanding the logic here?? Is there a way I can linked plans to run with a fairly large tolerance on when they actually start??

I hope I've explained my problem well enough??

Many Thanks,
Fraser Clarke