[BUG] Boomerang processing on iOS, portrait video issue.

[Craigtut]

Currently 2 issues with Boomerang processing on iOS: portrait video & exported file compatibility. I've spent hours trying to figure it out but having zero experience with swift or video processing I'm stuck at this point. I've tried implementing video orientation from here - raywenderlich - without much success.

I'd be very appreciative if someone with better experience could take a look at it.

Current Behavior

When using the boomerang processing on iOS, portrait orientation video is processed into a 90 degree rotated landscape video. It also doesn't play fully on any device but apple devices (the example video plays fine on Apple devices but won't in the browser)

Example of current iOS processing (This was a portrait video prior to processing):
https://firebasestorage.googleapis.com/v0/b/app-836pm.appspot.com/o/posts%2FaLiK84OPI7SxFLPYEJOUUKRpUo92%2FbDieykplBBQjB8qP450cIrJduov8WS%2F2018-12-16T08:30:49.210Z.mp4?alt=media&token=c0eaf5b6-7b44-41be-b8ad-3e0c4cf128bf

Expected Behavior

Expected to follow Android's behavior and process and export portrait video back into a portrait orientation that is playable on all devices.

Example of Android processing:
https://firebasestorage.googleapis.com/v0/b/app-836pm.appspot.com/o/posts%2FtvUWC2so22QhVfaNlHH2ielPrjB3%2FSXsyfT5O8yjG98rqf2h8RKotqKcqSB%2F2018-12-16T10%3A13%3A59.468Z.mp4?alt=media&token=49a4bc8b-4752-4769-93da-fff8be69475c

Your Environment

iOS 11 & 12

software	version
react-native-video-processing	1.7.1
react-native	0.57.3

[shahen94]

hi @Craigtut .

Thank you for reporting this issue, and I never experienced this before.

I can't promise that I'll have time to fix this, this library maintains community, so I'll mark this.

[Craigtut]

Thanks @shahen94 - any gut reactions with the playback compatibility issue below?

Still puzzled on the video compatibility. It's an odd situation where the full processed video plays fine on Apple devices, only the second reversed part plays on Android (first part glitches green), and only the first half part plays in Chrome then freezes.

Its exporting an mp4 so should be compatible everywhere. Maybe the codec used is the issue?

[shahen94]

@Craigtut I think so, some issue related with used Codec in AVExportSession.

[Craigtut]

Alright, a few half solutions if anyone else comes across this issue. I just don't have the experience to create a generalized solution and test across devices to make a PR.

If anyone stumbles here with similar issues, I ended up getting the video to rotate into portrait orientation using the following changes to RNVideoTrimmer.swift under the boomerang function:

    let mixComposition = AVMutableComposition()
    let track = mixComposition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: Int32(kCMPersistentTrackID_Invalid))
    
    // Change Starts - This is very specific to my needs
    var transforms = track.preferredTransform
    transforms = transforms.concatenating(CGAffineTransform(rotationAngle: CGFloat(90.0 * .pi / 180)))
    transforms = transforms.concatenating(CGAffineTransform(translationX: 640, y: 0))
    track.preferredTransform = transforms
    // Change Ends

    var outputURL = documentDirectory.appendingPathComponent("output")
    var finalURL = documentDirectory.appendingPathComponent("output")

As for video compatibility with Android, it seems that exporting with either AVAssetExportPresetPassthrough or AVAssetExportPresetHighestQuality seems not to use H.264. Hardcoding it to use AVAssetExportPresetMediumQuality apparently uses H.264 when exporting the video and can then be played on Android.

guard let exportSession = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetMediumQuality) else {
        callback(["Error creating AVAssetExportSession", NSNull()])
        return
      }

I think my specific issue is that react-native-camera records video with an HVEC codec as of iOS 11 so the first half of the boomerang won't play on Android then when just using AVAssetExportPresetPassthrough as it never gets converted to H.264. I might eventually try and mess with AVAssetWriterInput to manually specify the H.264 codec.

[BaderSerhan]

The issue is solved in #257