Ages ago I came up with the notion of interactive transcripts for Japanese video, so that you could click the text and the video would play from that spot; thus you could control the process both of engaged viewing and strategies such as sentence repetition (or ‘elicited imitation’) and shadowing. One of the strengths of text is the ability to externalize the words into a stable space which you can manipulate, allowing you to work at your own pace and refer back to previous areas of the text. Fortunately we have a tool, Kage Shibari—which I will give a tutorial on below, for extending this dimension to multimedia.
With interactive transcripts you are able to negotiate the pace of the audiovisual to your own level’s pace by subordinating it to the text. You can control the shadowing process and modulate the ‘push’ the multimedia engenders, turning it from a fluency exercise (which is best used for mastered materials, as emphasized in Paul Nation’s Four Strands) biased towards the aural and the macro level of the text, to a deeper, more cognitively involved exercise at the sentence level, according to the content and the aims of the learner. You can find related thoughts on this here.
It turned out there were already a few tools for this scattered here and there, in particular for interactive audiobooks. Using all of these ideas and their own ingenuity as a conceptual basis, the kind programmer known only by the moniker balloonguy thus created an efficient, minimalist .html file that could be used online or offline to play Japanese audio or video embedded in Japanese books and transcripts, where the relevant sections of the text would be highlighted as they play, and clicking on them would repeat the audio/video from that spot.
An original vision I had for such a resource with video was to focus on parsing the video on one’s own as much as possible before referring back to the subtitles, by watching without the subtitles on the screen but still available as a nearby tool. With Morph Man’s recent inclusion of adaptive subtitles that particular strategy may be obsolete, as now you can automate the process based on your tracked vocabulary knowledge, though this would locate any remaining difficulty one might have on lexicogrammatical parsing and speech perception, making the interactive text appealing once again.
Perhaps a combination of the two could be useful at some point, where text-based interactive navigation is retained. If before watching the video, you first pre-learn words and sentences in subs2srs Anki decks, then put it all together into a coherent whole outside Anki—which would mean watching the video as a continuous piece with all Japanese subtitles—then the strategy still applies.
At any rate, being mysterious and reticent, balloonguy did not name this program. So I dubbed it Kage Shibari (影縛り), referring to a fictional ‘shadow-binding’ technique (controlled shadowing).
I will now outline how to use a local, offline copy of Kage Shibari with Firefox or Chrome; Kage Shibari can be downloaded here. It supports various media formats (if necessary you can convert audio using the free program Audacity) and transcript/subtitle formats (.lrc, .srt, .ass, .trs). I will go over .trs files below.
1) If you want furigana, install Furigana Injector/HTML Ruby addons. For FI, check the ‘text in links’ box in the options.
2) Open the saved local .html of Kage Shibari in your browser. Chrome seems to play both .ogg and .mp3 better, whereas Firefox has issues using mp3 and is sometimes milliseconds off with .ogg; Chrome has exact timing using .mp3.
3) If you’re using an offline media file, next to Enter URL of Media paste file:///path (replace “path” with the path to the local file) and click Load.
4) Select Local beneath that and browse to the .trs file (.trs in this example, which indicates a transcript annotated with timestamps using Transcriber). You can also paste in the subtitles.
5) Right-click and select Furigana to inject the page with furigana, if desired.
6) Click play and watch/listen/read, clicking the text when you want to replay text or skip ahead to other areas that look more interesting (or easier, if you’re using proximal reading). Since it’s in the browser, you can also use Rikaisama or Rikaikun during this process.
- Transcriber is a great tool which is surprisingly fun to use, where you take any Japanese text and add timings to it. It creates .trs files, which are also supported in Subs2SRS. This is a tutorial from DingLabs, who later came up with an offline reader as well, here. After noticing Google Analytics code in the DingLabs Desktop Player last I checked, I refrained from using it, but your preferences may be less private. You can use Transcriber on audiobooks and their transcripts, such as here, or from entirely legitimate digital copies of Japanese readers which sometimes come with audio. I’m not sure if Read Real Japanese or Exploring/Breaking into Japanese Literature and similar books have legitimate digital copies, so you shouldn’t use those if you see them online someplace. Lastly, you could also request readers proficient in Japanese to record audio for brief swaths of text at RhinoSpike.
- A potential useful tool in the future would be an easy way to take English translations of the text and place them in popups grouped according to the highlighted Japanese text. Thus replacing the awkward parallel texts that some learners use.
- If you’re using Kage Shibari online, e.g. from Dropbox, according to balloonguy it can’t load local media, or, with Chrome, local subtitles—and remote subtitles must be from a publicly shared Dropbox.
