Tuesday, April 4, 2017

Recording screencast with Atom and Firefox

The need

I have recently started work on a video course about VueJS. The major part of it are screen recordings. I went through the original documentation my editor provided and it was just for Win/Mac (no surprise there) so I needed to go and figure it out for myself.

The how

As it turns out recording screen on Linux is quite easy. There are lots of good applications for it. I went with the good old ffmpeg

$ Xephyr :1 -ac -screen 1280x720 -br -reset &
$ sleep 2
$ setxkbmap -display :0 -print | xkbcomp - :1 &> /dev/null
$ DISPLAY=:1 evilwm -snap 50 &
$ DISPLAY=:1 atom &
$ DISPLAY=:1 firefox &

Now that we have all the apps running in a nice enclosure let's record the action:

$ ffmpeg \
    -f pulse -ac 2 -i default \
    -f x11grab -r 25 -s 1270x720 -i :1 -c:v libx264 \
This will create a new file with human readable timestamp each time you run this command encoding the 720p video in h.264 format and audio in AAC putting it all into an MPEG4 container.

Making Atom captures easy to edit

When editing the video in post production, for example using Blender's NLE, it is hard to fit the blinking intervals when cutting parts of the video. For that reason I recommend disabling the blink cursor in Atom. Select "Open your stylesheet" menu item and enter the following snippet

atom-text-editor.editor .is-focused .cursors .cursor {
  opacity: 1

That's it. Now the cursor is solid and editing is easy again

Happy recording!

Saturday, March 4, 2017

Sending logs from NodeMCU/Lua to Graylog2

Graylog2 is a great product. It might not be the most sophisticated one when it comes to log aggregation and interrogation, but it does its job very well. After employing Graylog2 to collect logs from all my Docker containers I thought that it would be absolutely fantastic to be able to manage logs from the IoT devices as well.

Installing the right input plugin

Before we can push stuff into Graylog2 we need to teach it the protocol of IoT devices - MQTT. Luckily enough there is a ready-made and supported input plugin just for that. To install it simply download it to the plugin folder in your Graylog2 installation and restart Graylog2. Done.

The next thing is to add an input for MQTT. Select System/Inputs and then create an input by selecting MQTT TCP (GEFL) from the drop-down list of available connectors. You need to specify the MQTT server and queue which the connector will subscribe to. In the example below I used system/logs so if you would go for a different name you need to adjust the code accordingly. And last but not least you need to start the input by pressing the Start input button. Done.

Programming in Lua

Lua and the NodeMCU environment is a beautiful thing. It is simple, powerful and it just rocks. For our little project we'll need to build ourselves a version with bmp085, cjson, wifi and mqtt modules. Then we need to download it to the module and we're all set to start coding in Lua.

The code here initializes WiFi connection, once the connection is established connects to the MQTT server, initializes the BMP085 sensor (here we're using pins D2-sda and D1-scl as per the documentation) and sets up a recurring alarm every second to read the sensor and send GEFL-formatted logs if MQTT has been connected. Easy.

Transferring files to NodeMCU

There are many great ways to send Lua files to the module. I like to use the one written in NodeJS called nodemcu-tool

$ npm install -g nodemcu-tool
$ nodemcu-tool upload init.lua
$ nodemcu-tool terminal

Now press the reset button and your NodeMCU should inform you of 3 things:

  1. That it connected to WiFi network and has an IP address
  2. That it has connected to the MQTT server
  3. That it is sending logs to MQTT server
There is not much more to it. Give it a try and if you stumble upon any challenges getting it to work feel free to leave a comment.

Happy logging!

Spring Boot application with Graylog2

I have been looking for an alternative to Splunk for a while and recently I have stumbled upon Graylog2. It is a very advanced tool well worth looking at. I decided to do a minimalist Spring Boot project to test its capabilities

The configuration

There are 2 steps required in order to get the logs to Graylog2:

  • Adding an artifact with GEFL appender
  • Adding the GEFL appender
I have tried 2 different GEFL appenders: Logback-gelf v0.10p1 which did work but didn't give me all the necessary details into Graylog2, and logback-gelf that worked for me very well.

I added the dependency to my pom.xml as follows


And then added a logback configuration file to my resources

<?xml version="1.0" encoding="UTF-8"?>
    <include resource="org/springframework/boot/logging/logback/base.xml" />

    <appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
        <layout class="de.siegmar.logbackgelf.GelfLayout">
            <shortPatternLayout class="ch.qos.logback.classic.PatternLayout">
            <fullPatternLayout class="ch.qos.logback.classic.PatternLayout">

    <root level="trace">
        <appender-ref ref="GELF" />  
        <appender-ref ref="CONSOLE" />

This gave me all the logs into a local instance of Graylog2 running on Docker.

version: '2'
    image: "mongo:3"
    image: "elasticsearch:2"
    command: "elasticsearch -Des.cluster.name='graylog'"
    image: graylog2/server:2.2.2-1
      GRAYLOG_PASSWORD_SECRET: somepasswordpepper
      GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
      - mongo
      - elasticsearch
      - "9000:9000"
      - "12201/udp:12201/udp"
      - "1514/udp:1514/udp"

Happy logging!

Monday, January 23, 2017

Creating an LTSP Server on Linux Mint

I wanted to try this out for ages but somehow I have never done it. Last weekend I decided to give it a go and to create an LTSP server and a thin client that'll start over PXE with no disk, floppy, USB stick or CD/DVD. It turned out to be quite simple but a few quirks made me wonder for a bit what the hell is going on. Here's a log of those things.

DHCP starts, TFTP from a live CD works but TFTP from PXE times out

That happens when using NAT network in VirtualBox. They are broken. Use 2 cards on the server, one in whatever mode you want, that will connect to the internet (I use bridged or NAT - both work fine) and for the second card use an internal network.

The server needs to have them properly configured. Leave the first one that's bridged to get its config using DHCP and the second one configure to have a static IP address (e.g., netmask of and NOTHING ELSE. For the client setup just one network card connected to internal network (same as for the server) and leave everything else be.

By the way, I used 2 different types of virtual network cards: Intel for the bridged one and AMD for internal network. The AMD card starts faster over DHCP :)

The /etc/ltsp/dhcpd.conf file needs to be updated accordingly. For the network I have configured mine as follows:


subnet netmask {
    option domain-name "example.org";
    option domain-name-servers;
    option broadcast-address;
    option subnet-mask;
    option root-path "/opt/ltsp/i386";
    if substring( option vendor-class-identifier, 0, 9 ) = "PXEClient" {
        filename "/ltsp/i386/pxelinux.0";
    } else {
        filename "/ltsp/i386/nbi.img";
I have removed option routers ... and next-server [...] options because they are not necessary as I am not connecting the client to Internet (everything runs on the server) and I am running TFTP on the same host with DHCP server.

On Linux Mint "Reached target Graphical Interface." but nothing happens

This one is easy but not obvious if you have done LTSP on Ubuntu. After the initial creation of image (ltsp-build-client [...]) install virtualbox' X11 guest driver like so:

$ sudo ltsp-chroot -p -d apt-get install virtualbox-guest-x11
And then update the image as usual:
$ sudo ltsp-update-image
On Ubuntu-based servers those drivers get installed automatically. Very weird...

Fucked up client display

If you will see a display that is sort of hacked in half horizontally and then vertically with things looking like it's Picasso's fantasy then your client has either too much graphical memory, 3D acceleration enabled or too few MBs of RAM. I set my client VM to have 1GB RAM, 128MB video RAM and 3D acceleration enabled but your mileage might vary. Before you do anything just add RAM - that should help

The client started - what's the username and password?

This one is really tricky if you don't know what LTSP is all about. Essentially LTSP client is Linux with X11 and SSH client bound to your server. So whenever you are logging in use the same account as on server. If you need more users - add them to the server. And remember: all apps are not running on the client - they run on the server! This is why a Raspberry Pi works great as an LTSP client. I love it!

I have checked Linux Mint XFCE, Ubuntu 16.04.1, LUbuntu, Ubuntu Mate - they all work great. If you're a RedHat/Fedora fan you're on your own.

Happy LTSPing!

Saturday, November 12, 2016

Flashing NodeMCU


This is going to be a quick one. When flashing NodeMCU with new firmware you need to remember that esptool.py requires some params for it to work correctly. And let's not forget the flash needs to be erased before we do anything.

esptool.py --port /dev/ttyUSB0 erase_flash
esptool.py --port /dev/ttyUSB0 write_flash -fm dio -fs 32m -ff 40m \
  0x000000 nodemcu-integer.bin \
  0x3fc000 esp_init_data_default.bin

Let me know if it works for you! Oh and btw - you can build the firmware online - I'm loving it!

Happy flashing!

Friday, November 4, 2016

Vue js - finally a framework I feel comfortable using

The story so far...

I have switched to frontend development about 10 months ago. I met a lot new friends, interesting ideas and got back to a place where you don't really need to travel half way around the world to react on someone pressing a button. That was great up until I was acquainted with JavaScript framework that I was supposed to work with. Ember.JS. I mean the idea behind it is great, don't get me wrong, but it is the tiny little details that make it a nightmare to use. Features like mixins, mustache syntax with helpers, views in God knows which location, lots of decisions pre-made for the developer (like we're not smart enough to make them ourselves, doh!) and the sick Broccoli souse on top of it - I hate Ember with a passion!

And so naturally I started looking for an alternative. First at AngularJS, then Angular2 - none of those felt like a good choice at the time. At some point I found a presentation where a scared to death person on the stage was criticizing MVC on the front end saying that's actually the wrong thing to do. That was a presentation of ReactJS. Funny enough a long long time ago I have had a conversation with one of my colleges about putting MVC to use in Delphi applications. It ended quite rapidly with me saying You're insane! and I have a strong feeling that it took 8 years for the industry to catch up with my statement.

But ReactJS felt a bit odd. And I don't mean the JSX syntax - I actually love it! But the whole shebang with Redux, functional components, state-full components - it was just a lot to take in. On top of that you needed to acquaint yourself with Webpack which at least fueled my interest in React for quite some time because of the hot module reloading. I even gave a talk about Webpack on a conference - that's how strong I feel it's a good thing

Let there be light (at the end of a tunnel)

And than, one evening, out of the blue someone posted an article on LinkedIn about vue.js. I never heard that name before, I gave it a quick read, got interested but not nearly enough to continue reading that night. Still some itch remained after I read that it was basically a simple framework. I can do simple - I actually prefer that over complex stuff - that is why I hate both Ember and JSF. A couple of days later I came back to it, read the whole user's guide in one evening and basically couldn't believe how obvious it was. I don't mean just simple (vue.js concepts are quite sophisticated at times) but just obvious. It takes all that's best from all the previous frameworks and does the same but in the right way.

Since then I have started a major project in vue.js, scaffolded with vue-cli, developed in Atom, with Java/Spring backend and I basically loved everything about it. The shorthand syntax for assigning event handlers with the @ syntax is just so fricken better than having to have 2 methods one for hooking up those events and one for unhooking them. The shorthand syntax for binding fields to views - awesome! The fact that you can have one file per component (not HAVE-TO but CAN) is such a relief from navigating millions of files in different locations in Ember. Using just properties and not custom-made this.get and this.set methods makes the code just so much more readable. But still - it's not the biggest win from using vue.js. It's how the framework structures and enforces things that makes it completely out of this world by comparison to other frameworks

A short tale of data and props

One of the biggest fuckups in all other frameworks (and in Ember in particular) is the fact that you create the API of your component when you use it. This basically means that there is no single point where you actually can do it. Not that it should bother you when you create the next "Hello, world!" but when it comes to a big project with 30 developers in it, having no definition of an API for a components and no enforcement of defining it turns on productivity after short few weeks.

Short personal note: I have seen a presentation from a very nice girl from the Polimer project at Google where she stated that if you're going to design a new component's API make it work like a button - not like the input.

vue.js actually makes you define the API or you can't use it! It even warns you that props should actually be read-only. You get a warning in the console when you try to assign a value to it!. How cool and thoughtful is that?!

Carry on my wayward son!

I think vue.js is the first of great many frameworks I came across that actually captures what developers need to have to do their job in a professional and maintainable fashion. Back in the days there was ExtJS which has fallen into the MVC trap as well (don't know the current state of things with that one though) but otherwise none of the currently popular frameworks feel like they are good fit for the job. So if you can - try vue.js out! And if your experience is not as mine - share it in comments! I'd like to know why and if that will impact me as well.

Have fun!

Thursday, October 6, 2016

Speeding up NPM

NPM is slow. I mean it is really, really slow when downloading packages. Partially because it makes a ton of requests to remote servers. That can be made faster!

There are 2 options that you'll find useful:

They both create a local cache that npm is later on interact with instead of the remote repository.


This one is really easy to setup and use. Just do
npm install -g npm_lazy
and use npm with the local registry like so:
npm --registry http://localhost:8080 install


This is a bit more complex. Nexus can be started as a stand-alone app quite easily but I would recommend using Docker as it is a far easier experience. So to start Nexus you type:
docker run -d -p 8081:8081 --name nexus sonatype/nexus
Then navigate to http://localhost:8081, login as admin/admin123 and create an npm-proxy. Select "Repositories" from the list on the left hand side, then from the "Add..." drop down at the top select "Add proxy repository", fill in as follows:
  • Repository ID - npm-registry-proxy
  • Repository Name = NPM Registry Proxy
  • Provider - npm
  • Remote Storage Location - https://registry.npmjs.org
and click on "Create repository".

Use it as follows:

npm --registry http://localhost:8081/content/repositories/npm-registry-proxy install

What's the gain?

Depending on the project you'll see a 30-50 percent speed boost when downloading packages. For the example project that I used here are the stats:
No proxy npm_lazy Nexus
1m43s 1m22s 57s

Have fun!

Edit on 23rd of January 2017: Forget all of the above. Just switch to Yarn and live happily ever after.