Building an External Scheduling Engine for D365 Project Operations | Part 7: Work That Arrives From Outside

Work that arrives from outside: integration-created orders, depth guards, date management and seeding

Everything so far has quietly assumed a planner. Someone opens a project, adds tasks, drags a bar, and the engine reacts. On a plant, that is the minority case. Most work orders are created by an integration from the maintenance system: a notification is approved, an order is released, and a project with its tasks appears in Dataverse with nobody at a keyboard.

If the engine does not run on that work, it does not run on the plant. This part is about the ways it silently fails to, and what a record that came from outside needs before it can be scheduled at all.

1) The guard that skipped the plant

Dataverse plugins run in a pipeline with a depth counter. A user saving a form is depth 1. A plugin that creates a record in response is depth 2; a plugin reacting to that is depth 3. The counter exists so a runaway loop eventually stops.

The most common loop protection in a scheduling engine is one line: if depth > 1, exit. It works. It also means the engine never runs on anything another process created, because an integration never operates at depth 1. A notification approval creates the project at depth 2 and its tasks at depth 3. A middleware drop lands at depth 2. Every one of them is skipped, with a trace line saying so and no error anywhere a planner would look.

The symptom is precise and easy to misdiagnose: everything created by hand schedules perfectly, nothing created by the integration schedules at all, and the two populations look identical on the form. In our case that one line explained the bulk of three separate defect reports before anyone read it.

2) What actually arrives

Design against the real payload, not the field list. A maintenance order from an EAM system typically shows up like this:

Three consequences for the engine:

• Every pass re-asks “can I place this now?” A handler that only runs on create, and needs the due date and the duration to be present on create, will never place an order whose due date arrives on pass two. Register the update handler for the same fields and make it do the same job. (Ours existed as code and had no registered step. It was reachable from nowhere.)
• Derive what is missing. Duration in days is the sum of operation hours divided by the template’s hours-per-day (Part 4), rounded up. Compute it when the payload lacks it; never overwrite it when the payload has it.
• Be idempotent. The same order will arrive three times. Placement, seeding and derivation must all be no-ops on a record that has already been placed.

3) Date management: from a due date to a plan

An order with a due date and a duration is enough to place it. The rule we use:

window        = workingMinutesBetween(today, dueDate)
plannedFinish = addWorkingMinutes(today, window × bufferFraction)   # fraction depends on work type
allowedStart  = subWorkingMinutes(plannedFinish, durationMinutes)
projectedFinish, exceedsDueDate = derived once tasks are placed

• The buffer fraction is a business rule. Preventive work may plan to finish well ahead of its due date; corrective work may use most of the window. Make it configurable per work type and do not bake a number into the code.
• Recompute when inputs change. A later pass that brings the due date, or a planner correcting the duration, should re-derive planned finish and allowed start. This is the update handler from section 2.
• Never move a placed schedule from here. Date management sets the envelope. It seeds tasks once (next section). It does not re-seed tasks a planner has since moved.

4) Seeding: a task with no start never enters the engine

This is the trap under the trap. After the guard is fixed and the envelope is computed, integration-born tasks still do not schedule, because every calculation in this series starts from a scheduled start, and they do not have one. The task-level logic computes a finish from a start; the critical path walks from early dates. Nothing seeds the first date, so nothing runs, and a project can sit with a perfect planned finish and six tasks with no dates on them.

seedTasks(project):
    cursor = project.allowedStart
    for task in tasksOf(project) ordered by operationNumber:
        if task.scheduledStart is set: continue          # idempotent: never touch a placed task
        if task has FS predecessors: cursor = max(cursor, latest predecessor EF)
        task.scheduledStart = nextWorkingInstant(cursor)
        task.manual = true                                # the engine treats it as placed
        write(task, tag = engineTag)                      # accepted by the task trigger at depth > 1
        cursor = addWorkingMinutes(task.scheduledStart, task.remainingMinutes)

• Run it when the envelope becomes available (due date pass), and on task create for tasks that arrive after the envelope already exists. Miss the second and any operation added later stays blank.
• Without dependencies from the source, “in operation order” is the sensible default sequence. Whether operations should actually chain finish-to-start is a question for the maintenance organisation, not the engine; make the default explicit and changeable.
• Seed in working time on the task’s own calendar. An order for a 24/7 unit and an order for a day crew should not land on the same instants.

5) The end task, again

Part 5 said the backward pass anchors exactly one end task. Integration-born orders are where that rule breaks, because the “which task is the end” flag is usually set by a rule like “if there are no other tasks yet, this one is the end”. If that rule counts only tasks that already have dates, every integration-born task sees zero dated siblings and every one of them becomes the end task. The critical path then anchors one of them at random and the rest drift.

Count all active siblings for the “first task” rule. Prefer the last operation in sequence as the end task. And when more than one is flagged, normalise to one, through the same write path that maintains the flag’s side effects rather than around it.

6) Ownership after arrival

The hardest decision in this part is not technical. Once the engine has placed an order and a planner has adjusted it, who owns the dates? If the source system sends a full update every time someone edits a description, and that update carries dates, the schedule is rewritten by a text change in another system.

• Dates flow in once, at create. After that the schedule owns them, and inbound updates map every field except the ones the engine owns.
• A small, explicit set flows back out. Typically a start date and a material requirement date, so the source system can plan procurement against the real schedule. Not the whole plan.
• Mark records the schedule has taken over, so a planner can see which orders will and will not be overwritten by the next inbound pass.

7) Test against real traffic

Constructed test data does not have the shape of real integration traffic: the missing fields, the pass ordering, the operations without dates. When the fix for the guard went in, orders kept arriving from the source system through the afternoon, and every change was tested against live drops rather than fixtures. When a later order exposed a defect the fix had introduced, it was caught on a real order within minutes. Build that loop into the environment: a way to replay real orders into a development instance, with traces on.

1. Create path: approve a notification with a required end date. Expect project, tasks, early and late dates, and no manual touch.
2. Later-pass path: an order whose due date arrives on the second pass places itself on that pass.
3. Manual regression: a project created by hand at depth 1 computes exactly as before.
4. Status-date regression: the sweep from Part 3 still pushes and recomputes, in comparable time.
5. Loop check: one orchestration run per change in the trace, tag-based exits on the engine’s own echoes, pending rows completing once each.
6. Ceiling: an artificially deep create chain hits the brake and corrupts nothing.

Summary

The engine that schedules a planner’s project and the engine that schedules the plant are the same code, but the second one has to survive a guard that would skip it, payloads that arrive in pieces, tasks with no dates, a rule that flags everything as the end, and a source system that wants its dates back. None of those show up on a demo. All of them show up in the first week of integration testing, and the difference between an engine that gets adopted and one that gets a spreadsheet next to it is whether they were designed for.

That closes the series as planned. If there is a Part 8, it is program-level scheduling: status-date ownership across sub-programs, portfolio roll-ups, and dependencies that cross project boundaries.