In this post I want to show you how to stabilize a video subject (person, object, etc.) with Kdenlive.
When I was trying to learn how to do this, I could only find tutorials on how to use vidstab
. However, that was not what I was looking for, because vidstab
tries to stabilize the whole frame, and it can introduce distortions in some areas.
The video below shows three different clips: the original footage (left), the stabilized clip obtained with vidstab
(middle), and the stabilized subject using my approach (right).
*The original clip used in this demonstration is available here.
We can see that vidstab
is able to stabilize some parts of the clip; notice how the car in the background is less shaky. But, in my opinion, the best clip is the one on the right, where the subject is stable and positioned at the center of the clip.
The first thing we need to do is to get the position of our subject at every frame of the clip. For that, Kdenlive has a neat effect called “Auto Mask”. Let’s go ahead and add that to our clip. (The clip should have been placed in the timeline by now.)
After adding the “Auto Mask”, a blurred rectangle should have appeared in the preview area; we are going to resize it and reposition it to roughly match the size and location of a feature point in your subject. In our case, the subject is a person, so we are going to resize the box to be slightly bigger than their face.
Finally, making sure the timeline cursor is positioned at the first frame of the clip, we are going to press the “Analyse” button in the options of the “Auto Mask” effect. This will scan each frame of the clip to track the face of our subject. Once the process finishes, a Tracking data link should appear in the “Auto Mask” effect. Click that link. The data has now been copied into our clipboard.
The image below summarises the steps required so far.
Now that we have the tracking data in our clipboard, we are going to invert it. This will allow us to later use it as a correction for the position of each frame. To make things easier, I have automated this part of the process with a bit of JavaScript. So go ahead and follow the steps below.
1. Specify the frame size of your project.
2. Specify an offset to adjust the position of the subject in the frame. (Negative values are allowed.)
3. Paste the tracking data of the "Auto Mask" effect below, then click on the button.
4. The new tracking data is now living in your clipboard. Try to paste the text with Ctrl + V in a different window to see the result.
Let’s go back to Kdenlive. We can now delete or hide the “Auto Mask” effect. Afterwards, we need to add a “Transform” effect, and use its options to import the keyframe correction data from our clipboard.
A small window should pop-up. We need to select "Position" from the dropdown, and then click OK.
And that is mostly it! If we playback the timeline, the clip should now be stabilised and focused on our subject. The only issue is that the frame is moving around, and we can see the black background of the project. A very easy workaround for this is to add a composition to our clip, and increase the zoom a bit, e.g., to 140%.
This is the end of the post. Thank you for getting this far! Feel free to leave a comment below.
]]>In this post I want to show you how easy and fun it is to model a little geometry quiz using Julia.
My goal is to give you a quick glance at the basic syntax needed to formulate and solve the problem.
Target audience: beginners in numerical optimisation or newcomers to the Julia ecosystem.
Consider the shapes shown in Figure 1 and the following constraints:
What is the area of the square?
You can try to solve this quiz on your own first, using a pen and a piece of paper.
There are many ways to reach the solution for this problem. The way I solved it was by realising that rectangles C and D are exactly the same. Why? We know all rectangles have the same area, and we also know that C and D share an edge. So, surely, they have the same height! That means that the height of B is two times the height of C or D. After knowing that, it is possible to solve for the height of B.
Now I want to show you how you could reach the solution by formulating a constrained optimisation problem. I am going to use JuMP, a language for modeling mathematical optimisation problems.
First, we import JuMP
and Ipopt
. Ipopt
is a free solver for these kind of problems:
using JuMP
using Ipopt
Afterwards, we create a new model and declare the problem variables (the height and the width of each rectangle):
model = Model(Ipopt.Optimizer)
@variables(model, begin
0 <= Ah ; 0 <= Bh ; 0 <= Ch ; 0 <= Dh ; 0 <= Eh
0 <= Aw ; 0 <= Bw ; 0 <= Cw ; 0 <= Dw ; 0 <= Ew
end)
Then, we give an initial guess for those values, and fix the value of the height of A:
# Specifying an initial guess
set_start_value.([Ah Bh Ch Dh Eh
Aw Bw Cw Dw Ew], 2)
fix(Ah, 2, force=true) # Given in the problem
Now, using the @constraint
macro, we define the problem constraints:
# Areas relationship: A = B = C = D = E
@constraint(model, Ah * Aw == Bh * Bw)
@constraint(model, Bh * Bw == Ch * Cw)
@constraint(model, Ch * Cw == Dh * Dw)
@constraint(model, Dh * Dw == Eh * Ew)
# Total area relationship
@constraint(model, 5 * Ah * Aw == Eh * Eh)
# Width relationships
@constraint(model, Cw == Dw)
@constraint(model, Aw == Bw + Cw)
# Height relationships
@constraint(model, Bh == Ch + Dh)
@constraint(model, Eh == Ah + Bh)
Notice that, in the constraints defined above, I never told the solver that Ch = Bh / 2
. I don’t have to! All of the constraints I declared are very straightforward, and directly observed from Figure 1—and that should be sufficient for the solver to figure everything out. How cool is that?!
Finally, we just need to call
optimize!(model)
Once the solver is done, you can print the answer to the quiz and the values found for every edge with:
area = round(value(Eh * Eh), digits=3)
println("Total area of the square is $(area).")
round.(value.([Ah Bh Ch Dh Eh
Aw Bw Cw Dw Ew]), digits=3) |> display
Which should output the following:
Total area of the square is 64.0.
2×5 Array{Float64,2}:
2.0 6.0 3.0 3.0 8.0
6.4 2.133 4.267 4.267 1.6
After having solved the problem, we can use the values to plot Figure 1 (the image shown on the top of this page) with the code below and using Plots.jl.
using Plots
rectangle(x, y, w, h) = Shape(x .+ [0,w,w,0], y .+ [0,0,h,h])
plot(axis=false, ticks=false, legend=nothing,
aspect_ratio=:equal, size=(600, 600),
background_color=:transparent,
foreground_color=:black,
palette=:Pastel1)
plot!(rectangle( 0, value(Bh), value(Aw), value(Ah)), linewidth=2)
plot!(rectangle( 0, 0, value(Bw), value(Bh)), linewidth=2)
plot!(rectangle(value(Bw), value(Dh), value(Cw), value(Ch)), linewidth=2)
plot!(rectangle(value(Bw), 0, value(Dw), value(Dh)), linewidth=2)
plot!(rectangle(value(Aw), 0, value(Ew), value(Eh)), linewidth=2)
font_size = 20
annotate!( value(Aw) / 2, value(Bh) + value(Ah) / 2, text("A", font_size))
annotate!( value(Bw) / 2, value(Bh) / 2, text("B", font_size))
annotate!(value(Bw) + value(Cw) / 2, value(Dh) + value(Ch) / 2, text("C", font_size))
annotate!(value(Bw) + value(Dw) / 2, value(Dh) / 2, text("D", font_size))
annotate!(value(Aw) + value(Ew) / 2, value(Eh) / 2, text("E", font_size))
annotate!(0.10 * value(Aw), value(Bh) + value(Ah) / 2, text("2", font_size))
quiver!([0.05 * value(Aw)], [value(Bh) + value(Ah) / 2], quiver=([0], [ value(Ah) / 2.1]), color=:black, linewidth=2)
quiver!([0.05 * value(Aw)], [value(Bh) + value(Ah) / 2], quiver=([0], [-value(Ah) / 2.1]), color=:black, linewidth=2)
savefig("./square_quiz.svg")
This is the end of the post. Thank you for getting this far! Feel free to leave a comment below.
]]>I used a Jupyter notebook to solve the reentry optimization problem with Julia and Knitro. If you are interested in trying to run this on your own machine, you can find the notebook in this GitHub repository.
The plots included below show information about the space shuttle as it reenters Earth’s atmosphere, from an altitude of 79,248 m (260,000 ft)
to 24,384 m (80,000 ft)
, and from a speed of 28,090 km/h (17,454 mph)
to 2,743 km/h (1,704 mph)
.
I also wanted to prepare a 3D animation to scale of the shuttle executing this trajectory using three.js, but my free time is over. I’ll keep that idea on the back of my mind.
Hopefully this was entertaining to you. Thank you for dropping by!
This is the end of the post.
]]>“Julia is a high-level, high-performance, dynamic programming language. While it is a general purpose language and can be used to write any application, many of its features are well-suited for high-performance numerical analysis and computational science.” — from Wikipedia: Julia (programming language).
In this guide, I’ll show you how to get started with Julia on Ubuntu 16.04 (or above).
Open a new terminal and go to your Downloads folder:
cd ~/Downloads
Use wget
to retrieve the latest compressed Julia Linux Binaries:
wget https://julialang-s3.julialang.org/bin/linux/x64/1.8/julia-1.8.3-linux-x86_64.tar.gz
Extract the .tar.gz
:
tar -xvzf julia-1.8.3-linux-x86_64.tar.gz
Copy the extracted folder to /opt
:
sudo cp -r julia-1.8.3 /opt/
Finally, create a symbolic link to julia
inside the /usr/local/bin
folder:
sudo ln -s /opt/julia-1.8.3/bin/julia /usr/local/bin/julia
Finally, you can test your installation by re-opening a terminal and typing:
julia
If you are here, chances are that you are just starting to learn Julia. I hope you enjoy the programming language as much as I have enjoyed it in the last couple of months.
If you are going to be using Julia in the context of optimization, check out this simple example: Solving a Geometry Quiz with JuMP. As a more complex example, you can use Julia to optimize the Space Shuttle Reentry Trajectory. Finally, for a real-world application of Julia, check out my research in robotics.
]]>Since my background is Software Engineering I did not want to push my first attempt too much, especially because my understanding of Electronics Engineering is very limited… Therefore, I decided to play it safe and look for an already existing kit of some sort.
Turns out that even though I got a kit, I still had to do some minor tweaking. It would have taken me much longer to get everything to work by trial and error on my own — as in, a lot more fried boards — and for that reason I am very grateful for the help of my friends and colleagues François Heremans and Wolfgang Merkt.
I started by watching some videos on YouTube of people that had already built their own prototypes. I ended up finding a seemingly-robust self-balancing robot called Balanbot created by a small Chinese company called Maker Studio. Here is their promotional video.
I made up my mind and ordered the kit from their website for $149.00. I placed my order on March 29 and the parcel arrived safe and sound on April 16, which is more than reasonable for a standard shipment from China to the UK.
The package contained everything required to build the robot and even brought some extra parts — which is very thoughtful given that there is nothing more annoying than buying a kit and getting stuck because a bolt has gone missing.
I was quite happy with the shipped FlyBlue + FlyDongle: a BLE wireless Arduino-compatible board, which allows for wireless sketch uploads. The way it works is very simple: power the Arduino, plug the FlyDongle to the computer, and voila: the connection is established.
I do have some minor “complaints”, however:
One of the copper spacers was a bit damaged on one of its ends, making it impossible to drive the matching screw through it — and from all the extra parts shipped, copper spacers were not amongst them;
The Balance Shield power button cap was missing;
The online product page is quite deceiving and had me assuming the robot would come with a LiPo battery — after all, every picture and promotional video of the robot features it drawing power from a LiPo battery.
The package did feature a battery case for 3x 18650 batteries — which can be bought from Samsung on Amazon for £16.99. I would expect that, for the kit’s $149.00 price tag, the batteries would have been included, though…
As a workaround for the missing batteries, I headed over to HobbyKing and ordered a 3-cell Zippy 2200mAh battery, a Turnigy LiPo battery charger, a voltage checker, and some XT60 connectors.
The HobbyKing parcel arrived in less than 48-hours since I ensured I was getting the items from the UK warehouse.
My next course of action was to make an adapter in order to connect the battery to the board. I used my very rudimentary soldering skills to solder the DC connector shipped with the kit to some wires, which in turn I soldered to one of the male XT60 connectors I got from HobbyKing. Finally, I applied a heat shrink tube over the adapter.
At this point I had all the key ingredients ready and I could finally start to build the robot. The instructions to build it are not very easy to find, but here they are.
Rather than screwing the battery case shipped with the kit, I used the zip ties to secure my LiPo battery in place.
After having finished assembling the robot, I fired up Arduino IDE and deployed the code supplied by the Balanbot makers (available here). It was a “success”. Well… at least for 20 seconds! Turns out the robot stopped working right after those glorifying seconds of self-balancing.
I think I actually fried the ATmega328P of the FlyBlue board — the Arduino-compatible board shipped in the Balanbot kit. I might have supplied a higher voltage to the board than it could handle. My bad…
I ended up buying yet another Arduino UNO R3.
Clearly, there was a problem on my power supply, which burnt my previous Arduino board.
In order to not repeat the same unfortunate situation, I had to make a more robust adapter to connect the battery to the Arduino board.
The input voltage of the new board was between 5V ~ 9V
.
As such, I ended up modifying the battery adapter according to the diagram shown in Figure 4:
8.5V
;The final result of the adapter is shown in Figure 5.
With these last tweaks I got the robot to operate once again and I have not had any further issues with it so far.
To end this blog post, I have put together a list of the individual components required to build the robot. The items are listed from lowest to highest price. As expected, the total price of the robot is actually less than the cost of the Balanbot kit.
I did not include structural elements of the robot (such as acrylic boards, screws, bolts) because I assume it will be easy for you to get a hold of those. And maybe you even have some of the items I listed lying around somewhere!
Price (£) | Item |
---|---|
0.99 | 12V DC Male Connector |
1.69 | 5pcs In Line 5x20mm Glass Fuse Holder |
1.95 | LM2596 DC-DC Buck Converter |
2.23 | LiPo Voltage Checker |
2.30 | 100pcs 5x20mm Glass Fuses Kit |
3.01 | XT60 Male/Female Connectors |
6.09 | Arduino UNO R3 |
9.33 | 3-Cell Zippy 2200mAh Battery |
10.12 | Turnigy LiPo Battery Charger |
11.79 | Balance Motor Driver Shield |
15.29 | 2x DC Gear Motors and Wheels |
64.79 | Total |
]]>Disclaimer: The prices in the table above might have changed since the publishing of this post.
I will keep this short and simple.
I used to have a somewhat-technical blog hosted at https://difusal.blogspot.co.uk/.
I have decided to revive my lost blogging activity here — since I prefer GitHub Pages — and now.
]]>