- VLCKit TestSuite
- 1. Add singular form for verbose time description
- 2. Setup testing schemes in Xcode
- 3. Tests: VLCTime
- 4. Create Rakefile
- 5. Setup CircleCI
- 6. Enable code coverage via xccov
- 7. Setup testing schemes for VLCKit
- 8. Setup testing schemes for TVVLCKitTests
- 9. Tests: VLCMedia
- 10. Tests: Add video samples
- 11. Tests: VLCAudio
- 12. Project organization
- 13. README: Reformat
- 14. README: Link build status badge with master
- 15. VLCLibrary: debugLogging status (closes #186)
- 16. Tests: VLCLibrary
- 17. VLCMediaThumbnailer: Update dimensions (fixes #183)
- 18. VLCMediaList: removeMediaAt (closes #182)
- 19. Tests: VLCMediaList
- 20. Tests: VLCMediaThumbnailer
- Moving forward
This summer, I participated in Google Summer of Code with VideoLan. My personal project was to establish the much needed test-suite for VLCKit, a generic multimedia library for audio or video playback needs on macOS, iOS and tvOS.
Before diving into the post, I would like to let you know that I will explain each pull request and why the said action was taken. This way, if you - the coherant reader - come by similar problems, you will know exactly what to do :D
Before the project started, VLCKit was already available to the public via source code and Cocoapods. However, even with over 80,000 downloads, and over 5,000 activate applications relying on it, the repo lacked a test-suite. My GSoC project aimed to solve this by integrating a test-suite along with a continuous integration workflow that would act as a safety net for any potential breakages.
Before listing all of my contributions, I would like to note that while my main task was to establish the said test-suite, I also decided to contribute to other projects within VideoLan during the coding period. You can read about them in the second part of the series.
Below is a list of pull requests that have been made throughout the GSoC coding period.
To be honest, the big code base kinda scared me in the beginning. To overcome this, I went and looked for the easiest, simplest looking class. After a bit of searching, I found
NSObject that represented the…. time. While browsing through the code, I found that its
verboseTimeDescription: function returned 1 minutes 20 seconds instead of 1 minute 20 seconds. It was really small but I had found my first bug 🎉
With @felix’s suggestion, I used Apple’s date component API. And voilà! My first PR was merged.
This was a simple PR setting up the testing scheme within Xcode. But the big lesson in this PR is this,
A pull request should only deal with ONE thing and ONE thing only.
This was the first PR that added tests to the codebase. After much discussion with my mentors, all tests written from this point followed the blackbox method of testing.
I wanted a script to expedite the process of building and testing. After having used fastlane not long ago in VLC-iOS, my mind went immediately to it. But considering fastlane has something like 20 dependencies, @caro thought that it would be too much when we were only going to use one of fastlane’s many features.
I agreed and began to look for lighter solutions. I eventually came across PSPDFKitSwift made by the good folks at PSPDFKit. In the repo, PSPDFKit uses a Rakefile to easily compile, and release the framework.
Inspired by this, I basically copied what they did… but with additional tasks for testing.
Lesson here? Never be satisfied with option #1. Look at other options and you will probably be surprised :D
This was relatively easy thanks to CircleCI’s thorough documentation. And with the
Rakefile from the previous PR,
.circleci.yml ended up looking clean.
I previously had used code coverage services like codecov and coveralls. So, I suggested VLCKit use one of these services. But after @caro mentioned that Apple had recently released
xccov, a code coverage report generator, I ended up integrating it in the aforementioned
While xccov is very easy to integrate, its output is too hard to read and not visually pleasing.
To solve this, I decided to create a xccov prettifier called xcperfect. Hopefully it can become a part of VLCKit once it becomes stable enough.
Simple PR to setup Xcode schemes 😵
Once more 🙃
The test here cover simple in & out style function with no internal mutations. The tests may look un-eventful, but make sure you cover all the edge cases. 😎
The question here was,
“How do we store test video assets in the repo?”
I will start by listing some of the options we considered and their pros & cons.
- Just store it in the repo
- Pro: Easy
- Con: Assets might get big and bloat the repo
- Pro: Easy to setup and buying into Git’s infra
- Con: Github requires you to pay 💰
- Pro: Easy to setup and buying into Git’s infra
- Con: “If you use git-submodules, you’ll have a bad time”
- Upload to server and download when building
- Pro: Large reliable storage is available for FREE
- Con: Relies on network to start initial test
Create a new repo called TestAssets and clone it via a Shell script during Xcode project’s initial build via RunBuildPhase.
11. Tests: VLCAudio
VLCAudio contains numerous internal state mutations. After initial doubts of whether blackbox testing was going work, this pull request proved that blackbox testing is going to work just fine.
The most satisfying thing ever is when git tells you…
13. README: Reformat
As @caro would say,
“People judge repositories by its… cover”.
Inspired by Swift’s Github repo, the README went through a major makeover.
To make the README even more attractive, a table with various status badges was added. This allows the users to immediately know the status of the repo along with the latest version of the framework.
When the user sets the
debugLoggingLevel/Status of their
VLCLibrary instance, no information was being saved nor checked. This PR fixed exactly that.
This implements tests for
VLCLibrary with the previous PR merged in. Tests here are pretty straightforward however, there are couple of functions that could not tested due their dependency on network calls.
This PR fixes a bug where the media thumbnail’s dimensions are not updated internally.
This PR fixes a bug detected via HockeyApp crash reports from the production release of VLC-iOS. There were many ways this bug could have been fixed. However, all methods shared a common theme
An API shouldn’t crash due to a potentially inconsistent state.
Think about this the next time you design your API.
Table-driven tests for the win!
XCTest doing some major heavy lifting in this PR.
There are still some test code that has not been pushed to the repository. To ensure that these get merged into the repo, I plan on keep submitting PRs to VLCKit in hopes to achieve 100% code coverage!
You have just finished part 1! Read PART 2 of the story for more eye-opening, panic-inducing, blockbuster action in the world of iOS.
« Xcode Auto Generated Interfaces
Google Summer of Code Final Report Part 2 »