Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ Some of this works today, and some of it is being built now. See [Status](#statu
| Make it your own | Its name, Bots, channels, model, and skills live in one folder, [`openbot/examples/openduet`](openbot/examples/openduet). | Available today |
| Watching mail, calendar, Slack | Notices new mail, calendar changes, and Slack messages as they arrive. Today it reads your mail when the brief is made. | In progress |
| What needs you, what can wait | Sorts new mail into what needs you and what can wait, with a one-line reason for each. Measured on a labelled set in [`evals/triage`](evals/triage). Slack comes later. | Available today |
| Morning brief | The first time you open Chat each day after your first, it is already there: what needs you and why, what can wait, today's calendar, and anything it could not check, posted once a day however many tabs or devices you open. You can also ask for it any time. **Start** on a row that needs you hands that email to OpenDuet. Getting the brief on a schedule, through routines, is not yet verified. | Available today |
| Tasks | One tab for what needs you, answered right there, what OpenDuet suggests from its background research, and what it is working on. Its newest suggestion also waits above Chat's composer, with **Start** and **Hide**. A suggestion you start stays in view: under **Working** while OpenDuet works, then **Ready for review** or **Needs input** with OpenDuet's next step, opening at its own message. Answer it in Chat, on its card or by deciding its approval, and it goes back to **Working** until OpenDuet replies. A run that fails says why, with **Try again** where running it again can help. Check one off and it folds into **Resolved**. | Available today |
| Morning brief | The first time you open Chat each day after your first, it is already there: what needs you and why, what can wait, today's calendar, and anything it could not check, posted once a day however many tabs or devices you open. You can also ask for it any time. **Start** on a row that needs you hands that email to OpenDuet as a task of its own, and the row reads Started on every device from then on. An app the brief could not read is named once, with the button that connects it. Getting the brief on a schedule, through routines, is not yet verified. | Available today |
| Tasks | One tab for what needs you, answered right there, what OpenDuet suggests from its background research, and what it is working on. Its newest suggestion also waits above Chat's composer, with **Start** and **Hide**. A suggestion you start stays in view: under **Working** while OpenDuet works, then **Ready for review** or **Needs input** with OpenDuet's next step, which it states at the end of each turn rather than leaving to the last line of its reply, opening at its own message. A task's report shows the pages it made. Answer it in Chat, on its card or by deciding its approval, and it goes back to **Working** until OpenDuet replies. A run that fails says why, with **Try again** where running it again can help. Check one off and it folds into **Resolved**. | Available today |
| Setup checklist | Onboarding asks you to connect Gmail and Calendar and to turn on notifications, each with **Not now**, then ends on a list of what OpenDuet depends on: Intelligence, the model key, Google, your Gmail and Calendar, notifications, and the optional extras. Each is **Ready**, **Not checked** or **Needs setup**, from the real answer, with how to fix it. **Check again** asks Intelligence and the model provider again. **Settings → Setup** opens it again. | Available today |
| Gets to know you | On your first day it speaks first: it greets you, reads your recent mail and this week's calendar, then answers in three short bubbles (who you are, three ways it can help, and an invitation to correct it), with **Sounds right** and **Something's off** to tap. With Gmail and Calendar not connected, it promises no mail: it says what it can do now and shows the card to connect them. | Available today |
| Make | Ask for a design system, a design, a dashboard, slides, a doc or a web page, or pick one from the **+** on the Make tab, and OpenDuet makes it. A card in Chat opens it, and Make keeps everything it made as thumbnails you can filter by kind. | Available today |
Expand Down
146 changes: 145 additions & 1 deletion openbot/app/src/components/channels/chat-messages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ import type { ActivityMessage, Message, ToolCall } from "@ag-ui/core";
import { classifyAttachment } from "@/lib/channels/attachments";
import { asText, REFUSAL_MARKER } from "@/lib/plugins/tool-result";
import { readChannelHello } from "@/lib/channels/channel-hello";
import { readTaskFrame } from "@openbot/shared/task-brief";
import {
MORNING_BRIEF_TOOL,
readMorningBrief,
} from "@/lib/channels/morning-brief";
import { readTaskFrame, TASK_STATUS_TOOL } from "@openbot/shared/task-brief";

/**
* Transcript projection that pairs assistant tool calls with later tool-result messages.
Expand Down Expand Up @@ -617,6 +621,12 @@ const SUGGEST_REPLIES = "suggest_replies";
const MAKE_PAGE = "make_page";
const OFFER_TASK = "offer_task";
const START_TASK = "start_task";
/**
* A task's turn saying where it stands, for Tasks (server/src/openduet/task-status.ts). It is for the
* list, not for the person reading the conversation, and draws nothing; kept as an item it would
* still take a row of the transcript's gap, so it is left out here.
*/
const TASK_STATUS = TASK_STATUS_TOOL;

/**
* How the runtime answers for a tool when it rejected the call's arguments before the tool ran, such
Expand Down Expand Up @@ -727,6 +737,8 @@ export function repliesLast(
sinceRefusal = null;
};
for (const item of items) {
if (item.kind === "tool" && item.toolCall.function.name === TASK_STATUS)
continue;
if (
item.kind === "tool" &&
(item.toolCall.function.name === OFFER_TASK ||
Expand Down Expand Up @@ -776,3 +788,135 @@ export function repliesLast(
close();
return ordered;
}

/** Words a stop can follow without ending the sentence: titles, initials' companions, Latin. */
const ABBREVIATIONS = new Set([
"mr",
"mrs",
"ms",
"dr",
"prof",
"sr",
"jr",
"st",
"mt",
"inc",
"ltd",
"co",
"corp",
"vs",
"etc",
"e.g",
"i.e",
"a.m",
"p.m",
"no",
"approx",
"dept",
"est",
]);

/**
* The text up to the end of its first whole sentence, or all of it when no sentence has ended yet.
*
* A sentence ends at a stop, a question mark or an exclamation mark, past any closing quote or
* bracket, followed by space and then something that can start a sentence; or at a blank line. Not
* at "Dr." in "Dr. Patel", "J." in "J. Smith", "e.g." or "3.5". Never inside a sentence: while one is
* still streaming in, all of it is shown.
*/
export function firstSentenceOf(text: string): string {
const body = text.trimStart();
const paragraph = /\n\s*\n/.exec(body);
const limit = paragraph ? paragraph.index : body.length;
const stop = /[.!?]["”’')\]]*(?=\s+["“‘(*_]*[\p{Lu}\p{N}])/gu;
for (const match of body.matchAll(stop)) {
const end = (match.index ?? 0) + match[0].length;
if (end > limit) break;
if (match[0].startsWith(".")) {
const word = /(\S+)\.["”’')\]]*$/u
.exec(body.slice(0, end))?.[1]
?.replace(/^[("“‘*_]+/, "");
if (
word &&
(/^\p{Lu}$/u.test(word) || ABBREVIATIONS.has(word.toLowerCase()))
)
continue;
}
return body.slice(0, end);
}
return body.slice(0, limit).trimEnd();
}

/**
* The Bot's words after a morning brief it posted by itself, held to the one sentence the frame
* asks for (shared/morning-brief.ts).
*
* FOUND ON A REAL BRIEF. The frame allowed "at most one short sentence" and the Bot wrote two, the
* second an offer of help, under a card that already puts Start on what needs the person. The
* frame's wording is tightened, and this keeps the screen to what it asks whatever the model writes:
* the first whole sentence after the card, never a sentence cut in half. Words written before the
* card are a greeting the frame forbids, under a card that already greets, so they are left out.
*
* Only on a turn the app asked for in the person's place: a brief the person asked for in their
* own words gets the Bot's whole answer. The thread keeps every word; this is what is drawn.
*/
export function briefSaysOneSentence(
items: readonly VisibleChatItem[],
): VisibleChatItem[] {
const out: VisibleChatItem[] = [];
let inBrief = false;
let carded = false;
let said = false;
// The turn's text items, so a turn whose card is followed by words can drop what came before it.
for (const [index, item] of items.entries()) {
if (
item.kind === "attachments" ||
(item.kind === "text" && item.role === "user")
) {
inBrief = item.kind === "text" && readMorningBrief(item.text) !== null;
carded = false;
said = false;
out.push(item);
continue;
}
if (!inBrief) {
out.push(item);
continue;
}
if (
item.kind === "tool" &&
item.toolCall.function.name === MORNING_BRIEF_TOOL
) {
carded = true;
out.push(item);
continue;
}
if (item.kind !== "text" || !item.text.trim()) {
out.push(item);
continue;
}
const cardLater = !carded && laterInTurn(items, index, MORNING_BRIEF_TOOL);
if (said || cardLater) continue;
said = true;
out.push({ ...item, text: firstSentenceOf(item.text) });
}
return out;
}

/** Whether a call to `name` comes later in the turn `index` is in, before the person next speaks. */
function laterInTurn(
items: readonly VisibleChatItem[],
index: number,
name: string,
): boolean {
for (const item of items.slice(index + 1)) {
if (
item.kind === "attachments" ||
(item.kind === "text" && item.role === "user")
)
return false;
if (item.kind === "tool" && item.toolCall.function.name === name)
return true;
}
return false;
}
7 changes: 6 additions & 1 deletion openbot/app/src/components/channels/chat-transcript.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ import { queryClient } from "@/query-client";
import { AppAccessCard } from "./app-access-card";
import {
attachmentModality,
briefSaysOneSentence,
repliesLast,
type SentAttachment,
toVisibleChatItems,
Expand Down Expand Up @@ -1960,7 +1961,11 @@ export function ChatTranscript({
*/
const thread = folded
? []
: groupToolSteps(repliesLast(toVisibleChatItems(messages)), isDrawn);
: groupToolSteps(
// The Bot's one sentence after a brief it posted itself; see `briefSaysOneSentence`.
briefSaysOneSentence(repliesLast(toVisibleChatItems(messages))),
isDrawn,
);
/*
* The first day's answer and its two replies. The replies only once the run has ended, while
* nothing the person said or parked comes after it, and only on a first day that read mail: they
Expand Down
64 changes: 50 additions & 14 deletions openbot/app/src/components/duet/morning-brief-card.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ export function MorningBriefCard({
canWait,
today = [],
notes = [],
quiet,
}: {
/** "Thursday, October 22". */
date: string;
Expand All @@ -56,19 +57,22 @@ export function MorningBriefCard({
canWait: BriefItem[];
today?: BriefItem[];
notes?: BriefItem[];
/**
* What the sources that were read and found empty came to, said under the greeting: "Nothing new
* came in to your inbox." Null says nothing. Absent, for a caller that cannot say which sources
* were read, the card says the old line only when every section is empty and nothing failed.
*/
quiet?: string | null;
}) {
const nothing =
needsYou.length === 0 && canWait.length === 0 && today.length === 0;
/*
* Where each section starts in the order the card builds in when it arrives live (build-in.tsx):
* the date and greeting first, then each section's label and the rows it shows, top to bottom.
*/
const sections = [
{ items: needsYou, label: "Needs you", tone: "accent" },
{ items: canWait, label: "Can wait", tone: "muted" },
{ items: today, label: "Today", tone: "muted" },
{ items: notes, label: "Couldn't check", tone: "muted" },
] as const;
const line =
quiet !== undefined
? quiet
: nothing && notes.length === 0
? "Nothing new needs you, and nothing is on your calendar today."
: null;
const sections = sectionsOf({ needsYou, canWait, today, notes });
let next = 1;
const starts = sections.map((section) => {
const start = next;
Expand All @@ -81,10 +85,8 @@ export function MorningBriefCard({
<BuildIn className="px-4 pt-[18px]" order={0}>
<p className="type-sub font-medium text-duet-ink-2">{date}</p>
<p className="type-display mt-0.5 font-semibold">{greeting}</p>
{nothing && notes.length === 0 ? (
<p className="type-sub mt-2 text-duet-ink-2">
Nothing new needs you, and nothing is on your calendar today.
</p>
{line ? (
<p className="type-sub mt-2 text-duet-ink-2">{line}</p>
) : null}
</BuildIn>
{sections.map((section, index) => (
Expand All @@ -102,6 +104,40 @@ export function MorningBriefCard({
);
}

type Sections = {
needsYou: BriefItem[];
canWait: BriefItem[];
today: BriefItem[];
notes: BriefItem[];
};

/**
* The card's sections in the order the card builds in when it arrives live (build-in.tsx): the date
* and greeting first, then each section's label and the rows it shows, top to bottom.
*/
function sectionsOf({ needsYou, canWait, today, notes }: Sections) {
return [
{ items: needsYou, label: "Needs you", tone: "accent" },
{ items: canWait, label: "Can wait", tone: "muted" },
{ items: today, label: "Today", tone: "muted" },
{ items: notes, label: "Couldn't check", tone: "muted" },
] as const;
}

/**
* How many beats the card takes to build in: the greeting, then every section's pieces. What is drawn
* under the card, such as the connect card for a source it could not read, arrives after this.
*/
export function briefPieceCount(
card: Pick<Sections, "needsYou" | "canWait"> & Partial<Sections>,
): number {
return sectionsOf({
today: [],
notes: [],
...card,
}).reduce((total, section) => total + piecesOf(section.items), 1);
}

/**
* The rows a section shows before "Show N more". A day of mail can sort dozens into "Needs you",
* and 27 rows on a phone are a second inbox, not a brief. Every row is still one tap away rather
Expand Down
38 changes: 36 additions & 2 deletions openbot/app/src/components/duet/task-report.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import { describeApproval } from "@/lib/approval-view";
import { ConversationPageCard } from "@/lib/copilot/page-card-tool";
import { type PageSummary, pageSummarySchema } from "@/lib/pages/queries";
import type {
ReportDecision,
ReportEntry,
Expand All @@ -11,10 +13,23 @@ import { DuetIcon, type DuetIconName } from "./icons";

/*
* A task's page, read as a report: when and why it was noticed (in the sheet's header), then
* Decisions and updates as a timeline, then Resources, the emails and events it read, then Show
* conversation, which unfolds the whole thread under it. The reply box stays at the foot.
* Decisions and updates as a timeline, then Made, the pages it made, each as the card Chat draws,
* then Resources, the emails and events it read, then Show conversation, which unfolds the whole
* thread under it. The reply box stays at the foot.
*
* MADE IS ITS OWN SECTION. A real report said "I made a page…" with nothing to open: the card was
* only under Show conversation, in the thread the report exists to save the person reading. What
* the task made is the thing they came to see, so it sits right under what it did.
*/

/** The report's pages that read as a page's summary; anything else is passed over, not guessed at. */
export function reportPages(report: TaskReport | undefined): PageSummary[] {
return (report?.pages ?? []).flatMap((page) => {
const parsed = pageSummarySchema.safeParse(page);
return parsed.success ? [parsed.data] : [];
});
}

const timeFormat = new Intl.DateTimeFormat(undefined, {
hour: "numeric",
minute: "2-digit",
Expand Down Expand Up @@ -131,6 +146,7 @@ export function TaskReportView({
<Timeline entries={report.entries} now={now} />
)}
</section>
<MadePages report={pending || error ? undefined : report} />
<section aria-labelledby="report-resources">
<ReportHeading id="report-resources">Resources</ReportHeading>
{pending || error || !report ? null : report.resources.length === 0 ? (
Expand Down Expand Up @@ -171,6 +187,24 @@ export function TaskReportView({
);
}

/** The pages it made, each opening the page; nothing at all for a task that made none. */
function MadePages({ report }: { report: TaskReport | undefined }) {
const pages = reportPages(report);
if (pages.length === 0) return null;
return (
<section aria-labelledby="report-made">
<ReportHeading id="report-made">Made</ReportHeading>
<ul className="flex flex-col gap-2.5">
{pages.map((page) => (
<li key={page.id}>
<ConversationPageCard page={page} />
</li>
))}
</ul>
</section>
);
}

function ReportHeading({ id, children }: { id: string; children: string }) {
return (
<h2
Expand Down
1 change: 1 addition & 0 deletions openbot/app/src/lib/channels/morning-brief.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
* `@/`, and the path to `shared/` is written down once.
*/
export {
BRIEF_EMAIL_SOURCE,
frameMorningBrief,
localDateOf,
MORNING_BRIEF_TOOL,
Expand Down
Loading
Loading