Kernel fails to load after Catalyst update

Today I did a system update on my Arch Linux installation with the command yaourt -Syua and after that the Kernel refused to boot with an error message similar to the following in journalctl -b:

kernel: RIP firegl trace+0x61/0x1e0

There is a know problem with the proprietary catalyst drivers not supporting linux kernels >= 4.2. So to fix this issue you have to configure the bootloader to use a different kernel, such as linux-lts. Install linux-lts package and then reinstall the catalyst* packages from the Vi0L0 AUR repos and then re-configure GRUB with grub-mkconfig -o /boot/grub/grub.cfg.

I had to boot and mount everything using the Arch live install system then chroot and reinstall the catalyst packages with the LTS kernel to fix the issue.

When rebooting make sure to pick the LTS kernel from the GRUB menu.

Vi0L0 himself has crated a bug for this at the Unofficial AMD Bugzilla, but a fix may take a long time if past behavior is any indication of current outcome.

Evaluating Ersatz Emacsen

I have been an Emacs user for many years and the key sequences have become ingrained muscle memory to me. Since most Linux or BSD distros come with vi installed by default, I have learned how to use it. But I still long for that familiar fell of Emacs under my fingers and, at the same time, wish there was something with a smaller footprint than full GNU Emacs.

Well, such software does exist, under many different implementations as a matter of fact. They are affectionately called Ersatz Emacs. Now that's a word your don't see every day, Ersatz. Search google for ersatz definition and you get the following:

er·satz

ˈerˌzäts,ˈerˌsäts/

adjective

(of a product) made or used as a substitute, typically an inferior one, for something else. "ersatz coffee" synonyms: artificial, substitute, imitation, synthetic, fake, false, faux, mock, simulated; More

not real or genuine. "ersatz emotion"

It seems that we can think of a an Ersatz Emacs as a fake emacs. And the Emacs Wiki pretty much confirms that:

A 'nonextensible imitation' of a supposed implementation of an Emacs;

The culprits

The most common Ersatz implementations of Emacs today are mg, jove, and zile. First let's learn a little bit about each of them.

Mg is a lightweight public-domain Ersatz Emacs, dating back to 1986. Mg was originally known as Micro GNU Emacs, as it strove to adhere more closely to the default behavior of GNU Emacs than other contemporary Ersatz Emacsen, but was renamed at the request of Richard Stallman. Modern versions of mg are maintained as a component of the OpenBSD source tree; the editor is included as standard in OpenBSD because there shouldn’t be any reason to learn more editor types than emacs or vi. There is also a port available for Linux.

JOVE (Jonathan’s Own Version of Emacs) is an open-source Ersatz Emacs implementation. JOVE is primarily intended for Unix-like operating systems, but also supports MS-DOS and Microsoft Windows. JOVE was inspired by Gosling Emacs but is much smaller and simpler, lacking any form of LISP or other extension language. It was originally created in 1983 by Jonathan Payne while at Lincoln-Sudbury Regional High School in Massachusetts, USA on a PDP-11 minicomputer. JOVE was distributed with several releases of BSD Unix, including 2.9BSD, 4.3BSD-Reno and 4.4BSD-Lite2.

GNU Zile ("Zile Implements Lua Editors") "is a text editor development kit, so that you can (relatively) quickly develop your own ideal text editor without reinventing the wheel for many of the common algorithms and data-structures needed to do so". Prior to version 3 its name expanded to "Zile Is Lossy Emacs". Its goal was to be a lightweight Ersatz Emacs from the Free Software Foundation emulating the behavior of its "big brother" in a small package, so that Emacs users can feel at home in limited environments. For customization, Zile uses its own limited configuration language known as Zile Lisp. Zile Lisp is a tiny subset of Emacs Lisp that consists of the Zile commands plus setq.

Comparison

The main goal is to pick an Emacs-like editor to be used for quick edits in a virtual host running a flavor of Debian Linux. To that end, I do not require any customisation of the editor, and I want the smallest footprint in both disk and memory.

Implementation First Appeared Latest Version Programming Language License
mg 1986 20150316 (linux port) C Public Domain
jove 1983 4.16.0.73 C Jove License
zile 2005 2.4.11 C GNU GPL

And for the technical details:

Memory
Implementation Size on Disk Virtual Physical Shared
mg 181 KB 10564 KB 2472 KB 2284 KB
jove 197 KB 8744 KB 2648 KB 2180 KB
zile 260 KB 47272 KB 3968 KB 2584 KB

The difference between Virtual, Physical, and Shared Memory

Time for a quick digression into what the different types of memory mean in Linux.

Virtual memory is the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card's RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. Virtual memory represents how much memory the program is able to access at the present moment.

Physical memory stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming.

Shared memory indicates how much of the Virtual memmory size is actually sharable memory or libraries. In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in Virtual and Shared, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under Physical.

Conclusion

zile is the largest on disk and biggest memory hog of the three, so that's and easy cut. jove is slightly larger on disk than mg, but it's Virtual memory size is the best. mg has the smallest footprint on disk and the smallest Physical memory footprint. Plus, all of the BSD's and all the major Linux distros have packages for the mg, so that's the one I am going with.

Renew IP lease on Linux

I run a router at home with dd-wrt. My main machine has a static IP lease based on the MAC address. Sometimes when I reboot the main machine the static IP is not properly released by the DHCP subsystem and the router holds on to it, therefore assigning my machine a regular DHCP IP.

To fix this without having to reboot the machine, follow these steps:

  1. Login to the dd-wrt router.
  2. Go to the Setup page and click the Save button followed by the Apply button. This will restart the DHCP daemon in the router without having to reboot it.
  3. Go to the Status page on the router and click on the Lan tab and verify that the IP was indeed released.
  4. On the main machine, run the following commands:
    sudo dhclient -v -r enp4s0
    sudo dhclient -v enp4s0
    

The above will renew the IP lease to the default static IP assigned to that MAC address. Note that the above assumes you are using NetworkManager with dhclient under it and that your network interface is enp4s0.

Get a prime number sequence using Java 8 Streams

With a spirit of expanding on my previous post about checking numbers for primality, here's a quick way to generate a sequence of prime numbers using streams:

public static LongStream primeSequence(long max) {
    return LongStream.iterate(2, i -> i + 1)
            .filter(x -> isPrime(x))
            .limit(max);
}

The above returns a LongStream of prime numbers up to a specified max value. It uses my previous prime checker function isPrime.

Hope it helps!

Check number primality using Java 8 Streams

The canonical way to check if a number is prime in Java uses a loop. It looks like this:

public static boolean isPrime(long x) {
    for (long n = 2; n <= Math.sqrt(x); n++) {
        if (x % n == 0) {
            return false;
        }
    }
    return true;
}

There is a more streamlined and efficient way of doing this using Streams. It looks like this:

public static boolean isPrime(long x) {
    return LongStream.rangeClosed(2, (long)(Math.sqrt(x)))
            .allMatch(n -> x % n != 0);
}

Hope this helps!

Install Brother HL-2270DW printer on Debian Jessie

I have a Brother HL-2270DW laser printer. Here's how to install support for it in Debian Jessie linux.

Install CUPS

First you need to install CUPS, the Common Unix Printing System, with the following command:

sudo apt-get install cups

Next make sure that the CUPS service is up and running:

systemctl status cups

Install the printer drivers

You will next need to get the Brother printer drivers from their website. Follow these steps:

  1. Go to http://support.brother.com/g/b/countrytop.aspx?c=us&lang=en
  2. Click on Downloads
  3. Search for HL-2270DW (my current printer)
  4. Select Linux for OS Family
  5. Select Linux (deb) for OS Version
  6. Click on Driver Install Tool for Printer Driver
  7. Agree to the EULA and Download
  8. Save it to a folder on your HDD and remember it
  9. Open a terminal window and go to the folder where you saved the printer driver
  10. Decompress the archive with

    gunzip linux-brprinter-installer-*.*.*-*.gz
    
  11. Become root by running the following command

    sudo -s
    
  12. Install the printer driver with the following command

    bash linux-brprinter-installer-*.*.*-*
    
  13. For Input model name enter HL-2270DW

  14. For You are going to install following packages. OK? enter y
  15. For Brother License Agreement Do you agree? [Y/n] enter y
  16. For GPL Do you agree? [Y/n] enter y
  17. Download of deb package and installation will start.
  18. for Will you specify the Device URI? answer n
  19. for Test Print? answer n
  20. Hit Enter/Return key.
  21. You are done with the Driver installation

Configure CUPS to use the printer

The final step of this process is to configure CUPS to use the new printer. Follow these steps:

  1. Open a web browser to http://localhost:631/
  2. Click the Administration tab then click the Add Printer button.
  3. Select IPP from the list.
  4. In the Connection field, type

    ipp://THE_PRINTER_IP/ipp/port1
    
  5. In the next form, give the printer a unique name (no spaces and the name be must unique from any identical printers), and select Brother from the printer make field.

  6. Select Brother HL2270DW for CUPS (en) from the list of drivers
  7. Configure the default options on the next page to your liking
  8. Set Duplex to DuplexNoTumble for double-side printing
  9. Set TonerSave to on to enable toner saving

That's it. Enjoy your new printer.

A note on migrating python virtual environments

The engine that builds this blog is called Nikola it's written in Python. I write the blog and run the Nikola compiler to build the site inside a Python virtual environment.

Up until recently I was doing this work on a machine running the latest Arch Linux and Python 2.7.8, but I have recently rebuilt that machine to run Debian 8, whose default Python interpreter is currently at version 2.7.6 and I copied my virtual environments to the new OS install.

This caused me a problem with my virtual environments that manifested itself as the following error when I tried to run any Python code from within the active virtual environment:

$ pip freeze
Traceback (most recent call last):
  File "/home/jonasg/.virtualenvs/tso/bin/pip", line 7, in <module>
    from pip import main
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/__init__.py", line 11, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/home/jonasg/.virtualenvs/tso/local/lib/python2.7/site-packages/pip/download.py", line 2, in <module>
    import email.utils
  File "/usr/lib/python2.7/email/utils.py", line 32, in <module>
    from email._parseaddr import quote
  File "/usr/lib/python2.7/email/_parseaddr.py", line 16, in <module>
    import time, calendar
  File "/usr/lib/python2.7/calendar.py", line 9, in <module>
    import datetime
ImportError: No module named datetime

$ nikola build
...
ImportError: No module named datetime

This happens because my virtual environments had a different version of /usr/bin/python2.7 (the one from Arch Linux) that — unlike the new binary — does not include the datetime built-in, and therefore generates an error when it cannot find it on disk anywhere. The new interpreter seems to import it without any file I/O (try running it under strace to check).

The fix for this issue is to activate the virtual environment you are having problems with and run the following:

$ cp /usr/bin/python2.7 $(which python2.7)

There are other ways to fix this issue, but I like the one above rather than re-initializing the virtualenv because I use virtualenvwrapper which does NOT gererate the virtual environment within the present directory.

Hope this helps!

Fixing package-query to pacman dependency in Arch Linux

I was recently trying to update my Arch Linux system and I ran into a peculiar dependency error with package-query that would complety halt the update.

error: failed to prepare transaction (could not satisfy dependencies)
:: package-query: requires pacman<4.2

It turns out that this is totally expected and it has to do with the new version of Pacman (4.2) breaking a few things related to backwards compatibility. This changes will require me to recompile and reinstall package-query and yaourt and here's how to do it:

  1. Uninstall package-query and yaourt

    sudo pacman -Rdd package-query
    sudo pacman -Rdd yaourt
    
  2. Get the latest bits from the AUR

    curl -O https://aur.archlinux.org/packages/pa/package-query/package-query.tar.gz
    curl -O https://aur.archlinux.org/packages/ya/yaourt/yaourt.tar.gz
    
  3. Explode the tarball and go into the each folder

    tar -zxf package-query.tar.gz
    tar -zxf yaourt.tar.gz
    cd package-query
    
  4. Compile

    makepkg -s
    
  5. Install

    sudo pacman -U package-query-1.5-2-x86_64.pkg.tar.gz
    

Make sure to do steps 4 and 5 for the yaourt package also. And this is it... All should be in order now.

Debian: Install Packages that were Kept Back

Have you ever seen this message when updating your Debian-based system: "The following packages have been kept back"? Here's why you see it and how to solve it:

You just tried to run the following command:

$ sudo apt-get update && sudo apt-get upgrade --show-upgraded -y

... and you get the message:

The following packages have been kept back

Running apt-get dist-upgrade is dangerous for a stable environment. The wrong source.list setting and you end up with a broken Debian. You might get the entire application upgraded to a version you don't want. For example, the kernel upgrade is kept back. You just want to upgrade the kernel, not the entire distribution.

A better way to handle a kept back package is:

  1. sudo aptitude
  2. If you have kept back package you should see Upgradable Packages on top of the list.
  3. Hit + on that list
  4. Hit g twice
  5. Answer debconf stuff if asked
  6. Press return to continue
  7. Press Q to quit
  8. Press yes

The kept back packages are now installed.

So, you want to write your own language?

Below are my notes from a really interesting article about the creation of programming languages by Walter Bright published on January 21, 2014 in the Dr. Dobbs Journal.

http://www.drdobbs.com/architecture-and-design/so-you-want-to-write-your-own-language/240165488

Work

Tons of work ... years of work ... need intrinsic motivation. Not a major cash investment

Design

Syntax matters a lot. It needs to be something that appeals to your target audience. A mix of familiar syntax + aesthetic beauty.

These should NOT be considerations:

  • Minimizing keystrokes
  • Easy parsing. not hard to write parsers with arbitrary lookahead
  • Minimizing the number of keywords

These things are TRULY GOOD:

  • Context-Free Grammar. code is parsable without having to look up in a symbol table
  • Redundancy. the grammar should be redundant to get good error messages
  • Tried and True. Stick with tried and true grammatical forms. cuts the learning curve. e.g. people expect + and * to follow a certain precedence rule.

After syntax, the important thing is semantic processing... meaning is assigned here to syntax structure. This is where you spend most of the design and implementation time. There is no glory in the semantics, but it is the whole point of the language.

After the semantice phase, the compiler does optimizations and code generation This is collectively called the back-end. Very challenging and complicated these 2 phases. Try using an existing back end such as one of JVM, CLR, GCC, LLVM.

Implementation

Regex is the wrong tool for lexing and parsing.

Don't bother wasting time with lexer and parser generators or other so-called compiler compilers. They are a waste of time. Writing the lexer and the parser is a tiny percentage of the job of writing a compiler.

Error messages are a big factor in the quality of implementation of the language. Use the following guidelines:

  • print the first error message and quit
  • guess what a programmer intended, repair the AST and continue
  • The poisoning approach. any operation with a NaN results in a NaN silently.

Compiler speed matters a lot... Most compilers are pigs. The secret to making a compiler fast is to use a profiler.

You must/have to be using these tools in order to be successful:

  • valgrind. saved C and C++ from oblivion
  • git / github like public repo. trasnformative tools. collaboration. track the origin of every line of code
  • Automated testing framework + Coverage analyzer
  • Automated documentation generator
  • Bugtracker + Wiki + Blog + Google Groups + Mailing List + IRC Channel + Twitter + Facebook

Lowering

A semantic technique. It consists of, internally, rewrting more complex semantic constructs in terms of simpler ones. e.g. while loops and foreach loops can be rewritten in terms of for loops. Then the rest of the code only has to deal with for loops. If there are special-case rules in the language that prevent lowering rewriting, then revisit the design of the language.

Runtime Library

Critical is the need to write a runtime library. This is a major project. It serves as the demostration of how the language features work. It better be good.

Get these right:

  • I/O performance. Slow IO will make that language look bad. the benchmark is C stdio
  • Memory allocation. Most of the time programs spend doing this
  • Transcendental functions... What are these? Find a library that already implements these functions ... Floating point stuff

What should go in a library

  • File IO
  • Network IO
  • String
  • Algorithms
  • Data Structures
  • OS Interaction
  • Unicode - UTF-8
  • Date / Time
  • Memory Management
  • Process Control
  • Signals
  • Math
  • Data Types
  • Crypto
  • Reflection

After the Prototype

  • Presentations
  • Articles
  • Tutorials
  • Books
  • Programmer Meetings
  • Conferences
  • Companies

Go anywhere they will have you and show it off. Get used to public speaking. Get as much feedback as you can.