Georgia Institute of Technology

On Demand and Streaming Video Captioning Project

DMI at Georgia Tech is a great system!  http://www.media.gatech.edu/

DMI has been disabled for direct access. All of the non-T-Square Data has been migrated to Mediaspace.gatech.edu (Kaltura). It now only supports the Digital Media tool in T-Square - until T-Square is shut down then it will be shut down also.

One of the things it cannot do is caption streaming and on-demand videos. The purpose of this project, site is to explore captioning of live video at Georgia Tech.

This project consist of this site with the latest JW Player, A server with Wowza Streaming Engine Perpetual Bundle Unlimited Edition and for streaming live events; both an EGG HD491 and  VidiU encoder - and some very clever behind the scenes work by the GT CATV RF Spec, OIT - Network Engineering group. We are also looking at the legacy DMI servers and exploring the possibility of captioning video on demand served from those.

News:

10/20/17

JW Player8  was installed today. Test feed is still on line today. Still no captions showing up for Kaltura Player. Did note today that Android Devices seem to display captions only on FaceBook Application Stream feed.

10/13/17

We had a server failure during a patch event which required a fresh Virtual Server and Application installation. I took this opportunity to upgrade to RHEL7 and beef up the server resources. After we went live captions were streaming nicely with test captions via new code with auto quality and dvr captabilty. Now, with the correct code, on webpages with JWP layer embedded, you will be able to move back up to one hour into the past if you tune in late to a broadcast. We are also able to push live video and captions from Wowza stream targets to FaceBook and YouTube pages. We can push video and audio to Kaltura - but it is still not displaying our captions.

Of course the test links are above but here they are again just in case you have trouble finding them (these are working as I write this, but of course require a live stream feed with captions and will not work unless it is there.): 

http://jwplayer.media.gatech.edu/oitmystream123/oitmystream123-playlist-dvr-apple-ios-device-apple-hlscupertino

10/22/16

The team has done some great work and we are now able to stream live video with captions using our hardware/software to JW Player and YouTube. You can see some of the captioned recordings under the VOD link above. We can also push the stream from our Wowza server to Kaltura and Facebook, but for now the 608 embedded captions do not display there. Please look at this page: Live Event Kaltura iFrame embed. Kaltura is the Future of streaming media, live and on-demand, at Georgia Tech and everyone at Tech should use it to live stream video with captions.  All content from the legacy DMI system should be migrated to Kaltura Mediaspace by January 2016.

If you request and are granted permission by the mediaspace team, anyone with access can create a live stream today from Add New> Live Event after logging into mediaspace.gatech.edu. The  Live Event Kaltura iFrame embed page above will show a live feed when it is streaming and a recording of the live feed after the event is over if the resulting recording is changed from private to unlisted. Subsequent recordings on the same live event append the recording so it may make sense to clip the recording and save it, then delete the recording before the next live event. Captions on the recording are machine generated and added to the recording after the live event. This page shows my first attempt to embed the Kaltura iFrame in a Drupal page. This probably can be finessed by Drupal expert. The out of the package page on mediaspace.gatech.edu works better than my current embed: https://mediaspace.gatech.edu/edit/1_juxvpy8q. A work flow for the best way to produce and manage Kaltura live streams still needs to be developed.

I discussed  live streaming captions with Kaltrua and one of their partners,  Uvision (videolinq http://www.videolinq.com/).  It appears that today we could live stream captioned Katura video by using the videolinq cloud service with our existing EEG 491 hardware encoder. Other USG schools may be able to do this also if they subscribe to EEG Falcon Cloud (https://eegent.com/falcon) Please see the workflow from the vendor below.

Unfortunately, some recent staffing changes have forced me to reduce my effort around this project and I will have to leave the testing and implementation of the live stream captioning capability of Katura up to others, for now.

7/28/16 - Today with the help of the folks at JW Player we solved the auto-display captions requirement. Now the McCammish Control and OIT_MyStream123 pages on this site have the new code that auto displays captions when live streaming. Captions are displaying automatically in YouTube Live Events http://b.gatech.edu/2aAydVD ;  but only in on the iPhone and iPad in the Kaltura player http://b.gatech.edu/2ay4NpH

7/1/16 - Today we successfully pushed the live feed from Wowza to Kaltura and YouTube. Even though this test feed had captions enabled we were unable to see them on the respective players. We will test again this month with a test production feed.

We have been doing some initial work with the Kaltura project team to investigate the migration of video content in DMI to Kaltura. The link below is to a file that was imported directly from DMI in to Kaltura. We also added an srt captioning file to the video.

https://cdnapisec.kaltura.com/index.php/extwidget/openGraph/wid/1_wyt5qp3m

The current production GT live streaming service is now being supported only by stream.media.gatech.edu (wow04). 

Wow! The weekend of 6 and 7 May, 2016 was a banner weekend for the project! We provided emended captioning in the live video feeds of commencement ceremonies:

http://jwplayer.media.gatech.edu/sample-graduation-2016-vod-wcaptions

Useful code for embedding live streams in Drupal: http://drupal.gatech.edu/embed-live-streaming-video-oit

We are continuing to look at accessibility issues. Here is a demo page that shows a captioned video on demand using an accessible player: http://jwplayer.media.gatech.edu/ccplayer.html. Note how all controls are accessible from the keyboard (use tab key to access). This is built with ccPlayer from NCAM. http://ncam.wgbh.org/invent_build/web_multimedia/tools-guidelines/ccplayer

Not all of the links above are active all the time:

1. VOD - You can see a basic Video On Demand (VOD) from this link. This video is served from wow04 and played on the JW Player on this site. This video shows a recording of one of our early test of live stream captioning.
2. local video - You can see a basic video from this link. This video is served from this site and played on the JW Player on this site.
3. CaptVideo - You can see a sample captioned video from this link. This video is served from this site and played on the JW Player on this site. You can always turn the captions on and off with the red "CC" button on the player. This button is only available when the player detects captions.
4. OIT_MyStream* links - You can see different resolution feeds from the EGG HD491 when they are available. The CEA-608 captions are injected into H264 SEI messages of an HTTP Live Streams. It is processed by wow04 and played by the JW Player on this site.
5. WedVTT Caption of DMI VOD - You can see a WebVTT captioned video from this link. This sample VTT file was created with https://developer.microsoft.com/en-us/microsoft-edge/testdrive/demos/captionmaker/ and is served from this site. The video is a standard DMI video played from the DMI repository and both are played with the JW Player on this site. This Video On Demand also demonstrates the captions on by default and auto play options in JW Player.
6. WebVTT Caption of VOD - You can see a WebVTT captioned video from this link. The VTT file is served from this site. The video is is served from wow04. Both are played with the JW Player on this site.
7. mmcammish control - You can see the feed from McCamish Pavilion when it is playing. We hope to live caption ceremonies held here.This stream uses the VidiU encoder and streams through wow04 before it is played by the JW Player on this ste.
8. i video - You can see the feed from GoCoder, a wowza live streaming phone application, when it is running. This stream plays from a phone, is served from wow04 and played by the JW Player on this site. You should be able to download GoCoder onto your IPhone or Android phone and experment with live streaming if you have a streaming account on wow04.
9. VOD w/captions - You can see a video on demand served from wow04 with captions also served from wow04 and played by the JW Player on this site. The Wowza server and JW Player work the magic of combining them and making them available. When we figure out how to allow DMI users to upload a same-named *.vtt file to the same directory as the video it is subtitling then we will have a way to caption any video on demand that is loaded into DMI. Wowza on the production DMI servers would need to be updated to make this possible.

JW Player

• PLAYS EVERYWHERE, EVERY TIME https://www.jwplayer.com/​
  • The most-used and fastest loading video player on the web, our signature product enables online publishers to deliver and monetize video across all devices in any browser. From the video blogger to the Fortune 100 in need of an enterprise-grade solution, JW Player lets users create the best video experiences for viewers on the web, mobile, and OTT devices.
• STREAM EVERYWHERE WITH HLS AND DASH
  • Provide your viewers peak quality without buffering, saving them bandwidth. And with DRM in HTML5, you can better protect your content
• YOUR VIDEOS ON EVERY PLATFORM
  • Your videos play on any device your viewers choose—website, smartphone, or Chromecast on TV.
• CONTROL YOUR BRANDING AND UX
  • Don’t just embed video in another brand’s player; brand and style your player your way, with CSS skins and JavaScript that let you control the user experience.
• CUSTOMIZE WITH A JAVASCRIPT API
  • Develop and build a unique experience with more event callbacks and built-in methods than any other video player.

• Adding Closed Captions:

  • JW Player 7 supports the rendering of closed captions or subtitles in the video display. These captions can be selected by viewers through the CC button. Here is an example, using a single language. The CC button then acts as a toggle:

    • For multiple languages, the CC button will show a language selection menu. Click here to load multiple languages for above example video (or toggle back to single CC).​

    • Supported Formats

      • JW Player 7 currently supports captions in two formats:
        • WebVTT, a plain text format that's part of the HTML5 standard.
        • VTT is required to render tracks on iOS Fullscreen
        • SRT, a plain text format that's popular amongst file sharers.
      • VTT captions are strongly preferred, since they are part of the HTML5 standard and will work across more browsers (Namely iOS in full screen). JW Player supports both formats in non-fullscreen in both Flash and HTML5 modes. As of JW 7.3, playback in fullscreen on iOS can be accomplished when using VTT captions. The player will intelligently attach a VTT track element when Safari is detected.
      • JW Player also supports captions embedded in the stream for HTTP Live Streaming.
      • CEA-608 Captions
        • The JW Player will detect and display CEA-608 captions that were injected into H264 SEI messages of an HTTP Live Streams. These captions can be turned on or off with the normal closed captions toggle. The detection of CEA-608 captions is available in the Premium edition of the JW Player.

Wowza Streaming Engine

• Any Media to Any Device, Anywhere https://www.wowza.com/
• Multi-Protocol, Multi-Client Delivery
  • Save time and money by using a single live stream to simultaneously deliver live video and audio streams to any player, any device, over any protocol.
• Live Streaming from Any Encoder
  • Use the best encoding solution for your needs, from free software-based RTMP encoders to broadcast-grade MPEG-TS hardware.
• Live Streaming from IP Cameras
  • Broadcast directly from H.264 IP cameras to connected devices, record live streams for instant DVR or later on-demand playback.
• Support for the Latest Standards
  • Deliver streams using popular formats like Apple HLS as well as emerging formats as they come on the market, including MPEG-DASH.
• On-the-Fly Transcoding
  • Transcoding means re-encoding one video/audio file format into another and converting one bitrate/resolution into another for compatibility across all devices and bandwidths. Transcoding with Wowza™ technology lets you refine your live encoding and streaming workflow:
  • Use multiple bitrates to reach devices with lower processing power & bandwidth.
  • Avoid up-front costs with unlimited transcoding and pay-as-you-go pricing.
  • Convert between formats to deliver appropriate streams to each device.
• Flexibility and Control
  • Live, Linear, and On-Demand Streaming
  • Create live, linear, or on-demand streaming applications such as live events, news, surveillance, training, and on-demand videos.
• PTZ Camera Controls
  • Control the pan/tilt/zoom (PTZ) functionality of certain Sony and Axis IP (network) cameras.
• Robust REST and Java APIs
  • Configure, manage and monitor custom integrations and implement custom modules.
• Multiple-Language Support
  • Reach global audiences with video streams that allow viewers to select the audio or captioning from multiple languages.
• Live and VOD Caption Conversion
  • Deliver live and on-demand videos with captions that support multiple scenarios and comply with FCC regulations.
• Live Stream Recording and Archiving
  • Record a live stream to a file available for later on-demand playback. Record an entire live webcast into a single file, segment it into multiple files for chapter replay, or start and stop recording at predetermined points for partial archiving.
• Engaging Viewer Experiences
  • Live Adaptive Bitrate Streaming
  • Provide the best possible quality for each viewer by adjusting video stream delivery in real time according to viewers’ varying CPU and bandwidth using adaptive bitrate streaming.
• Surround-Sound Audio
  • Create the ultimate immersive experience with surround-sound audio.
• Pause, Replay, Fast-Forward, & More
  • Create a premium viewing experience by leveraging capabilities such as pause, play, instant replay, rewind, and fast-forward, using Wowza nDVR.
• High Performance and Scalability
  • Dynamic Content Retrieval
  • Scale on-demand streaming by dynamically retrieving content from a web server or network file system.
• Origin Serving
  • Scale live and on-demand streaming across large regions and audiences by using Wowza Streaming Engine as an origin server for CDNs and more.
• Connection Management
  • Automatically disconnect idle clients to clear inactive connections.
• Live Stream Distribution
  • Use Stream Targets to scale and add redundancy to your live streaming workflow. Send an incoming live source stream to the Wowza Streaming Cloud™ service, third-party CDNs, YouTube, etc.
• Dynamic Load Balancing
  • Maximize performance by dynamically balancing connection loads among multiple Wowza Streaming Engine instances.
• Multicast and Unicast Streaming
  • Make effective use of enterprise networks with unicast streaming across a multi-server public or private network, or multicast streaming across large, colocated groups of viewers on private networks.