Repository navigation
Conversation
Ports the tested Kids Mode screens (src/KidsMode/design, verbatim, types stripped) onto this app's stack: track-player audio, vector icons, bundled art, react-native-audio-recorder-player recording and a scoring client that uploads nothing unless the parent opts in. Kids entry in the home header, progress persisted in the kidsMode slice, scorer URL and key as config in constant.js.
avioberoi
changed the title
Register Feather and FontAwesome5 fonts on iOS so the Kids icons render; drop the duplicate Baloo entries. Record AAC mono on both platforms. Say plainly when no scorer is configured; retry only dropped connections. Ignore stale audio loads and skip reloading an already loaded track. Leave the journey through the navigator instead of the design's copy of the home screen; keep the design's preview controls to dev builds; state the real privacy terms. Remove unused images, styles and strings.
A new storage object on every render made the design's save effect re-run, and each save re-rendered the screen through the store, so Kids Mode looped on mount. Story art is re-encoded at 1600 px from the originals instead of 1024, as the story scene pans across it at four times the screen width.
Android's back button now steps through the journey like the header chevron. The status bar follows Kids night mode instead of staying white. A closing ॥ stays on the line of its last word. Switches share one track colour.
…reciter Sections, text, transliteration and translation come from the bundled database and line timings from the reader's per-reciter lyrics, so any bani with a recording can load; Japji is checked identical to the design's data on all 374 lines. Settings gains a Reciter row; audio and scoring follow the choice. The scoring client takes the bani id from the loaded bani instead of a constant.
The practice label, the privacy line and the word review name the reciter the parent chose instead of Bhai Jarnail Singh. The screen waits for the bani before mounting the design and shows a database error with a way back. A manifest track by a reciter the scorer has no reference for is skipped rather than mislabelled. The Japji copy of the design's data is gone now that the database supplies it.
|
The feature looks amazing, especially UI! The journey, practice flow and scoring client are well put together, and the opt-in and "nothing uploads unless the parent turns it on" handling is careful. I'm requesting changes for the recorder lifecycle and two scoring correctness issues. Points to review1. The microphone can stay on after Kids Mode is closed. 2. The challenge's finish logic can run twice. 3. A line the service doesn't cover can never be passed. 4. The score can be against a different reciter than the one shown. 5. No handling for backgrounding or audio interruptions. UX Fixes
2. Children see raw technical errors
3. Night mode is incomplete
|
…in load error Inside the app, every path to the design's own home screen (bottom bar Home, back from settings, its Kids Mode switch) leaves the journey instead, and that switch is hidden since the parent's lives in Settings. The sheet and the loading and error screens follow night mode. A database failure shows a plain message and logs the detail.
…as such The new region environments replace the six earlier images, which nothing references now. A line the service could not check reads "Not checked" instead of a percentage.
Work begun before a cancel, an unmount or a move to the background stops at its next await instead of reopening the microphone. stop() is single-flight and the design's finish guard is a ref, so one line is checked once. A line the service cannot check is skipped when it begins, left out of the average and shown as not checked, instead of being recorded forever. The score is always against the reciter the screen names; a reciter the service lacks blocks checking with a plain message. Backgrounding cancels the recording and sends the child back to ready; an empty or missing file is never posted. The hooks return the same object until a field changes. Technical detail goes to the log; the child reads plain words.
# Conflicts: # src/HomeScreen/components/BaniHeader.jsx # src/HomeScreen/styles.js # src/Settings/index.js # src/common/actions/index.js # src/common/localization.js # src/navigation/index.jsx
The bani query lives in the database module with the other queries and logs its failures. The header's Kids glyph is an IconAction like the settings glyph opposite it. The error screen's way back is the app's Button with its Go back string. A recording under a kilobyte is treated as no recording. The review test asserts the fields it cares about instead of a snapshot. The six region scenes are re-encoded to the size the other art uses.
|
Thanks, Manjot! All eight are in now, each with a test that fails on the old code, and the branch also takes in dev at 806bdab and Gurjaap's new region art.
Testing: lint and the full suite (66 suites, 984 tests) pass on the merged tree, the Android build runs the whole Kids Mode flow on an emulator in CI with screenshots, and Gurjaap is doing the iPhone pass for the recorder and playback paths. The branch carries two merge commits and one of Gurjaap's, so a squash merge would keep dev's history clean if that's your preference. Let me know if anything else needs a change. -Avi |
What this adds
Kids Mode: a practice layer for children in grades 6 to 8 that sits on top of the banis Sundar Gutka already carries. A bani becomes a journey of sections; each section has listen, practise and a recited challenge with word-level pronunciation feedback against the reciter the child chose. Japji Sahib ships first with a full designed journey. The child listens to a reciter, recites a line, and sees which words sounded like the reciter and which need another try. It is off by default. A parent turns it on in Settings, and a second switch has to be on before any audio leaves the phone.
We built this for the Sarbat hackathon as an extension of Sundar Gutka, so the aim was to touch as little of the existing app as we could and keep the new code in one folder.
How it is wired
SettingsSectionandSettingsToggleRow, with the Kids Mode switch, the "Check pronunciation online" switch, and a Reciter chooser (Giani Gurdev Singh, Bhai Jarnail Singh, Bibi Indermohan Kaur). Audio and scoring follow the choice.KidsHomeroute with the native header hidden.src/KidsMode/design/: the screens as our designer built and tested them in an Expo app, ported to JavaScript with the types stripped. They stay verbatim (/* eslint-disable */at the top) so they can be diffed against the source. The edits we did make are listed below.src/KidsMode/host/: what the screens need from the app, written for this stack: reference audio through react-native-track-player, icons through react-native-vector-icons, bundled art, and line recording through react-native-audio-recorder-player.src/KidsMode/scoring/: the client for the scoring service.KIDS_SCORING_URLandKIDS_SCORING_API_KEYinsrc/common/constant.jsare empty here. With no URL the online check stays off, and with the opt-in off the client returns without a network call.kidsModeredux slice (experience) and persist with the rest of the store.src/database/db.js(getBaniLinesByToken), line timings from the reader's per-reciter lyrics files, the track from the audio manifest. Japji built this way is identical to the design's own data on all 374 lines. The screens open Japji, whose journey the design draws; the other banis in the catalog stay "coming soon" until they have one.RECORD_AUDIOon Android;NSMicrophoneUsageDescriptionand the Feather and FontAwesome5 icon fonts inUIAppFontson iOS (the Baloo Paaji 2 fonts were already registered); one new dependency (react-native-audio-recorder-player3.6.14).How the scoring works
The app knows the text, so the service does not transcribe. It aligns the recording to the expected text (CTC forced alignment), then scores each word on how strongly the model's frame probabilities favour the expected sounds, against the one reciter the child heard. The text it aligns to is the recited form, as the terminal sihari and aunkar are written but not spoken, and all three reciters in the app drop them (33 of 36 words in Pauri 1). Each word comes back as
like,checkorunclear. There is no 0 to 100 score per word, as the acoustic distances mean nothing at that resolution. Silence, a clip under a second, or more than half the words unaligned returnsunclearwith no score, as we would rather say nothing than invent a number.We tried five models on the three reciter recordings that ship with the app, with planted errors (matra swaps such as ਸੋਚੈ to ਸੋਚਿ, consonant swaps such as ਲਖ to ਰਖ, a deleted word) and timed them on CPU:
kdcyberdude/w2v-bert-punjabi)surindersinghssj/indicconformer-pa-v3-kirtan)The Gurbani-trained IndicConformer runs the demo service, as it is fast and Apache licensed. It cannot hear the silent-sihari rule, so w2v-BERT Punjabi is the candidate for production once it has an int8 export or a GPU. No model heard a dropped word-final kanna (ਸਿਆਣਪਾ to ਸਿਆਣਪ) or extra speech the text does not contain. Everything was measured on three adult professional reciters in studio conditions. No child's voice has been scored yet, so the false-flag rate on children is unknown. The live service scores Japji, Jaap, Savaiye, Chaupai, Anand, Rehras and Sohila against the reciters the CDN has for each, and serves the word clips the review screen plays. Method, scripts and numbers: https://github.com/Gurjaapk1/Ucharaan-2026 (
docs/scoring-writeup.md,demo-server/).Edits to the designed screens
KidsApptakeshost.initialScreen, so the Kids icon opens the journey (or the explorer choice on first run) instead of the design's own copy of the Sundar Gutka home.BackHandlerlistener).components.jsx); the stored text and the word tokenizer are untouched.host.reciterName) instead of a fixed name.host.exitinstead of the design's own copy of it.__DEV__).Where this goes
The pieces are per bani, not per Japji. Lines and sections come from the app's own database, line timings from the reader's per-reciter timing files, reference audio from the CDN, and the scoring service takes a bani id. Adding a bani to Kids Mode is content work: a map, section art and a story for it. Pronunciation feedback is the first thing a child gets from it; the longer aim is that practising a bani daily is something a child wants to keep doing, which is what the explorer, coins and rewards are for.
Files outside src/KidsMode
Testing
yarn lint: 0 errors, 1 warning (the existingno-consoleinindex.js).yarn test: 66 suites, 984 tests pass on the merge withdevat 806bdab.like; with line 3 cut out, every word of line 3 ischeckand the line scores 0; half a second of silence returnsunclearwith no score. Requests take 1 to 3 s.devat 806bdab): the app launches, Kids Mode and online checking are switched on in Settings, the Reciter chooser opens and closes, and a Maestro script walks explorer → worlds → map → Pauri 1 → practice (reference audio plays) → challenge → Pauri 2 story → night mode and back out, with a screenshot at each step. Fonts, icons and art all render in day and night mode. On this build the challenge screen says "Pronunciation checking is not set up in this version", since no scoring URL is committed. This ran on a copy of this branch with test IDs added for the script.Review round (manjotkaur27, Oct 18)
All five recorder and scoring points and the three UX points are addressed, each with a test that fails on the old code; the reply on the PR lists them one by one. Child-facing text is in
src/KidsMode/scoring/messages.js; technical detail goes tologError.Known limits
src/KidsMode/host/bani.jskeeps its section splitting and seq ranges keyed by bani token on purpose: the data layer is per bani even though only Japji has a journey.Kids Mode copy is English only, like the ported screens; the child-facing strings sit in
src/KidsMode/scoring/messages.js, and localising them is a follow-up.The line timings in
bani.jsonend a little early on four of the six Pauri 1 lines, so "Listen" clips the last word; a data fix for the designer.Reference audio and word clips go through the app's single track player, so opening a line in Kids Mode replaces whatever the reader had queued.
Playback after recording on iOS depends on the audio session being reset, which needs a device check. A phone call that never changes
AppStateis not detected on Android (the recorder library exposes no interruption event).Scores are uncalibrated heuristics from a one-day spike; the 10% false-flag target is a target.
The bani loader reads the audio manifest directly, so the reader's offline downloads and its length-variant tracks for Chaupai and Rehras are not used yet; Japji is unaffected.
Licence
Code under the repo licence. Baloo Paaji 2 is under the SIL Open Font License; GurbaniAkhar was already in the repo. The artwork was made by our team for this feature.