Skip to content
Merged
2 changes: 1 addition & 1 deletion packages/junior-dashboard/src/mock-reporting/fixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1451,7 +1451,7 @@ function eventAutomationRunConversation(
"This is a task, not a message from a person.",
"",
"About: Issue · ACME-42",
'Created by: <@U123>. "Me" and "my" in the instructions mean this person: mention them in the message. If the instructions do not say "me" or "my", do not mention this person.',
'Created by: <@U123>. This person is the Automation creator. "The Automation creator", "me", and "my" in the instructions mean this person. Where the instructions refer to this person, write this mention in the message. If they do not, do not mention this person.',
"Instructions: Summarize the closed issue",
"",
"Trusted summary: ACME-42 Checkout fails for saved cards was closed by Morgan Lee.",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ describe("Schedule Creation", () => {
expect(creates[0]!.input).not.toHaveProperty("nextRunAt");
});

test("when a reminder for the requester posts to the channel, name the requester exactly", async ({
test("when a reminder for the requester posts to the channel, save it for the Automation creator without conversation references", async ({
run,
}) => {
const conversation = await run(
Expand All @@ -90,10 +90,11 @@ describe("Schedule Creation", () => {
conversation,
);
expect(creates).toHaveLength(1);
// The task runs later without this request, so "me" must become a mention.
// The run does not see this request, so "this channel" would mean nothing.
const { instruction } = creates[0]!.input as { instruction?: string };
expect(instruction).toContain("<@U0TEST>");
expect(instruction).toMatch(/\bAutomation creator\b/i);
expect(instruction).not.toMatch(/\b(?:me|my)\b/i);
expect(instruction).not.toMatch(/\bthis channel\b/i);
});

test("when asked to tell the channel something later, preserve the future work in the schedule", async ({
Expand Down
17 changes: 17 additions & 0 deletions packages/junior-evals/evals/integration/scheduler/delivery.eval.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,23 @@ describe("Scheduled Delivery", () => {
expect(delivery.replies[0]!.text).toContain("<@U0TEST>");
});

test("when a due reminder names the Automation creator, mention its creator", async ({
run,
}) => {
await insertScheduledAutomation({
credentialMode: "system",
destination: slackChannel(),
due: true,
once: true,
task: "Remind the Automation creator to do healthchecks.",
});

const delivery = await run(heartbeat());

expect(delivery.replies).toHaveLength(1);
expect(delivery.replies[0]!.text).toContain("<@U0TEST>");
});

test("when a person replies to a delivered reminder, answer in its thread as a normal chat reply", async ({
run,
}) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,34 @@ describe("Schedule Management", () => {
expect(update.input).not.toHaveProperty("nextRunAt");
}
});

test("when a requester who did not create a reminder adds themselves, name them with their mention", async ({
run,
}) => {
const channel = slackChannel();
await insertScheduledAutomation({
createdBy: alice,
credentialMode: "system",
destination: channel,
task: "Post a planning reminder in this channel.",
});

const conversation = await run(
slackMention(
"change the scheduled planning reminder now so it also reminds me to bring the agenda.",
{ channel },
),
);

const updates = completedToolCalls(
"slackScheduleUpdateAutomation",
conversation,
);
expect(updates).toHaveLength(1);
// Each run reads "me" and "the Automation creator" as Alice.
const { instruction } = updates[0]!.input as { instruction?: string };
expect(instruction).toContain("<@U0TEST>");
expect(instruction).not.toMatch(/\b(?:me|my)\b/i);
expect(instruction).not.toMatch(/\bAutomation creator\b/i);
});
});
2 changes: 1 addition & 1 deletion packages/junior/src/chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ so it has no instructions of its own. `agent-invocations/result.ts` owns its
| 1 | `[task]` | yes | Task header. Same for schedule, event, and subscription. |
| 2 | Origin | yes | `This is a task, not a message from a person.` |
| 3 | `About:` | no | One-line resource label. |
| 4 | `Created by:` | no | Creator mention, and the rule to mention them only for "me" or "my". |
| 4 | `Created by:` | no | Creator mention. "The Automation creator", "me", and "my" mean them. |
| 5 | `People involved:` | no | Event people mentions. Guidance says when to address them. |
| 6 | `Done by:` | no | Linked person who did the Event, tied to its provider handle. |
| 7 | `Instructions:` | yes | Stored task text or subscription intent. |
Expand Down
15 changes: 11 additions & 4 deletions packages/junior/src/chat/automations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,17 @@ instruction or event selector switches creator credential use to system
credentials. The edit read returns `ownedByViewer` so the editor can show
these rules before a save.

An Automation runs later without the request that created it. The agent tools
that create or update an Automation reject an instruction that names the
requester by display name without their Slack mention. A name does not notify
the person.
An Automation runs later without the request that created it. So the agent
tools tell the agent to write exact targets, such as a channel mention or a
link. The agent calls the creator "the Automation creator". It names every
other person with their Slack mention, also a requester who did not create the
Automation. Each run puts the creator mention in the `Created by:` task input
line. That line also says that "me" and "my" mean the creator. People can
write these words in the dashboard editor.

The agent tools that create or update an Automation reject an instruction that
names the requester by display name without their Slack mention. A name does
not notify the person.

## Versions

Expand Down
7 changes: 4 additions & 3 deletions packages/junior/src/chat/automations/edit-schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ export const automationTitleSchema = z.string().trim().min(1).max(60);
export const automationInstructionSchema = z.string().trim().min(1).max(4000);

/**
* Instruction input for agent tools. An Automation runs later without the
* request around it, so the instruction must name each person exactly.
* Instruction input for agent tools. The run does not see the request, so the
* instruction names exact targets. It calls the creator "the Automation
* creator". It names every other person with their mention.
*/
export const automationInstructionToolSchema =
automationInstructionSchema.describe(
"Work to perform when the Automation runs. It runs later without this conversation, so write it to stand alone: name each person with their Slack mention, such as <@U123>, instead of me, my, or you. Preserve the user's scope, targets, and constraints. Do not add permissions or side effects the user did not request.",
'Work to perform when the Automation runs. The run does not see this conversation, so words like "this channel" or "here" will mean nothing then. Write the exact target instead, such as a channel mention or a link. Call the person who created the Automation "the Automation creator", not "me"; each run knows who that is. Name every other person with their Slack mention, such as <@U123>, including a requester who did not create the Automation. Preserve the user\'s scope, targets, and constraints. Do not add permissions or side effects the user did not request.',
);

export const automationEditFieldsSchema = z.object({
Expand Down
5 changes: 3 additions & 2 deletions packages/junior/src/chat/task-input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ export function renderTaskInput(args: {
outcomes?: TaskOutcome[];
/** Human label for the matched resource, when present. */
about?: string;
/** Mention of the person who created the task, so "me" in it is exact. */
/** Mention of the Automation creator, so "me" and "the Automation creator" are exact. */
creator?: string;
/** Mentions of the people involved in an Event. Guidance says when to address them. */
people?: readonly string[];
Expand Down Expand Up @@ -103,9 +103,10 @@ export function renderTaskInput(args: {
"This is a task, not a message from a person.",
"",
...(about ? [`About: ${oneLine(about)}`] : []),
// Say what to write, or the model drops the mention.
...(creator
? [
`Created by: ${oneLine(creator)}. "Me" and "my" in the instructions mean this person: mention them in the message. If the instructions do not say "me" or "my", do not mention this person.`,
`Created by: ${oneLine(creator)}. This person is the Automation creator. "The Automation creator", "me", and "my" in the instructions mean this person. Where the instructions refer to this person, write this mention in the message. If they do not, do not mention this person.`,
]
: []),
...(people.length > 0 ? [`People involved: ${people.join(", ")}`] : []),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ describe("event automations", () => {
This is a task, not a message from a person.

About: GitHub PR getsentry/junior#1174
Created by: <@U123>. "Me" and "my" in the instructions mean this person: mention them in the message. If the instructions do not say "me" or "my", do not mention this person.
Created by: <@U123>. This person is the Automation creator. "The Automation creator", "me", and "my" in the instructions mean this person. Where the instructions refer to this person, write this mention in the message. If they do not, do not mention this person.
Instructions: Address the requested changes.

Trusted summary: A reviewer requested changes.
Expand Down
2 changes: 1 addition & 1 deletion packages/junior/tests/unit/chat/task-input.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ describe("renderTaskInput", () => {
This is a task, not a message from a person.

About: GitHub PR getsentry/junior#691
Created by: <@U123>. "Me" and "my" in the instructions mean this person: mention them in the message. If the instructions do not say "me" or "my", do not mention this person.
Created by: <@U123>. This person is the Automation creator. "The Automation creator", "me", and "my" in the instructions mean this person. Where the instructions refer to this person, write this mention in the message. If they do not, do not mention this person.
People involved: <@U456>
Done by: <@U456>. This person did what this Event describes. The Event calls them "octocat". That is the same person. They are also in People involved. Write to them as "you", and do not also call them "octocat".
Instructions: Fix failed checks on this PR.
Expand Down
Loading