Repository navigation
feat: Summary, Transcription, Quiz, Subtitle from AI-generated Transcription - #81
Open
Punnawit9285 wants to merge 17 commits into
Open
Punnawit9285 wants to merge 17 commits into
Punnawit9285 wants to merge 17 commits into
Conversation
Two cards below the Pomodoro timer on the study page, filled from FlickMan's v1/study_aids API: - Summary: the lecture as a checklist. Sections are highlighted and jump the video to where each topic starts, points are numbered straight through, and the points a reader ticks are kept on the device. - Quiz: single-best-answer questions one at a time, showing the answer, the explanation, how the class did and a link back to that part of the lecture. Editors chosen on FlickMan can rewrite the summary and add, remove or correct questions, and decide whether readers see their name as a co-author.
Native checkboxes and radios follow the system's light or dark scheme, so on a light theme under a dark system they showed as dark squares.
… link - Admins get an Editors screen on the summary and quiz cards: add someone by their @docchula.com address as an editor or admin, change roles, remove them. FlickMan keeps at least one admin. - Editors can add images to a summary and to a question or its explanation. Large photos are made smaller before upload; only images stored on FlickMan are ever loaded, so a pasted image cannot track readers. - The quiz card has a Share button. Its link (?video=<id>&quiz=1) opens the lecture with the quiz ready.
Each card gets a button that opens a large window, like the transcript's, with Summary and Quiz tabs. On wide screens the summary's sections are listed beside it. Jumping to a part of the lecture from the window closes it first, so the video can be seen. The cards' contents become summary-body and quiz-body, shared by the cards and the window, and ticks and quiz answers move into a service so both show the same progress.
…e study aids The summary and quiz cards become one "Summary & Transcription" card with Summary, Transcription and Quiz tabs; the reading window has the same tabs. - Transcription: FlickMan's word-for-word AI transcript, each paragraph with the time it starts, which plays the lecture from there. The paragraph being spoken is marked, and the list can follow the lecture. - Captions: a button in the player's control bar turns captions from the same transcript on and off. FlickMan only knows when each paragraph starts, so a paragraph is cut into two-line captions spread over its time. - Verified by: editors and admins mark the summary or quiz they have checked against the lecture. Readers see who verified it, that nobody has yet, or that it changed since. - Readers can switch between the edited version (shown first), the AI's original and a Changes view that marks every word added or removed, letter by letter for a corrected spelling, coloured by the editor who made it. - Editors have a display name, shown when they are credited, when they verify and beside their changes. FlickMan allows changing it once every 30 days, so the co-author box no longer takes a name with each save.
FlickMan now serves the study aids through Video.studyAids, Video.transcript and its study-aid queries and mutations instead of v1/study_aids; only the images keep their own addresses. Saves and verifications read the outcome FlickMan reports (saved, unchanged, conflict or missing), and refusals and validation messages come from the GraphQL errors' extensions. An image goes up as a GraphQL multipart request.
…ptions button - Pictures: editors paste a Google Drive link to a picture shared as "Anyone with the link" instead of uploading it, so FlickMan neither stores nor serves images. Flick checks the picture loads before inserting it, saves only the link and draws the picture from Google when a reader gets to it. - Ticks: the summary points a reader ticks are kept on FlickMan as well as on the device, so they show on the reader's other devices. Changes go about 1.5 s after the last tick, and a device that was offline sends its ticks on the next visit. - Captions: video.js's own captions menu is removed, so one button turns captions on and off, underlined while they are on. - Transcription: lines FlickMan stamps one by one are grouped into reading paragraphs, and each line plays the lecture from where it is said.
The results list cut every question after two lines, so the end of a long question was hidden. It now wraps to its full length.
Redo quiz now shows on the start screen once some questions are answered and beside the progress dots while answering, as well as on the results, where it replaces Retake quiz. While the quiz is unfinished it asks first, since it clears the answers given so far.
A diagram drawn in a code block, such as a flow of veins, lost its shape in the summary: each line had a grey strip of its own, the first line sat a little to the right of the rest, the tall line height broke its vertical lines apart, and on the card its right side could not be reached. - A code block is one box in a fixed-width font, its lines close enough for │ to join up, and it scrolls sideways when the card is too narrow. - The Changes view keeps an edited diagram's lines and spacing. - The summary conventions name code blocks for diagrams.
Punnawit9285
changed the title
The study aids should not depend on a team keeping them up over the years, so everything that needs people goes, along with the captions: - Editing the summary and the quiz, the editors list for admins, verification, the history of every version with what each editor changed, crediting editors and their display names. - Pictures from Google Drive, which only editors added. An image in the Markdown is now always shown as a link. - Captions from the transcript, which live in their own branch now. What is left is the AI's work, read as before: the checklist summary with ticks that follow the reader across devices, the quiz with class statistics, redo and a link to share it, the transcript with its timestamps and download, and the reading window. The card is marked "AI-generated", and the summary and quiz say they were written by AI from the lecture recording.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a "Summary & Transcription" card under the Pomodoro timer on the course page, with up to three tabs: the lecture as an AI checklist summary, its AI transcript, and a quiz. Editors chosen by admins can edit/correct/verify the summary and the quiz, and the player gets captions from the transcript. Everything comes from FlickMan through GraphQL (the FlickMan PR from the
study-aids-adminbranch).I know this might indeed add complexity to the codebase with (96 files added/change) and might be difficult to review, but unlike settings feature in the earlier PR, I think this actually worth the complexity added. This is what my friends actually asked for including subtitle feature.
For editors and admins part, I'm a core team of Peer Support myself. They all love the new "AI transcription" feature P"Keen just added but suggested that it would be very nice if PS can verify / edit them.
With this they can now edit or add note/quiz as they want along with adding timestamp where does the information come from in the lecture manually (they can add images too using google drive link too).
Please consider adding this feature kub 🙇♂️