C# things that make us happy

Sometimes our lives as programmers get a bit easier, Microsoft has launchedlaunched C# 7.0 some time ago. Here are a few examples of the things that made our lives a lot easier.

The microsoft page has better explanations, the code below is just random code from my project. These examples are only intended to outline some of the new features.

Pattern matching

Can be rewriten as:

Using the underscore we can ignore variables, which keeps the compiler (and resharper) happy:
DoSomething(out String variable1, out String _)
Which brings us the pleasure of inline variable definition as well as ignoring unused ones.

It can also create funny code (introducing underscores in useless places):

Tuples
While we have had basic tuples for a long time they grew up and gave us a weapon against the useless class/structs to perform returns (or way to many out variables).

Keep in mind that a tuple is a struct!

Switch statements with patterns
While switching still isn’t available on primitive types it sure improves our lives a lot:

The nature of C# locking issues explained

Sometimes in the life of a programmer everything seems peachy, C#/DotNet provide relatively easy multi threading capabilities which 99.9% of the time work just fine as explained everywhere online.

However when things get a bit serious as these online explanations tend to be off by a small bit, only that small bit creates havoc….

Imagine having to debug code within a lock() statement that crashes with exceptions like “Collection was modified; enumeration operation may not execute” which seemingly bypass any locking in place.
These are not fun to deal with as everyone will for some reason stubbornly tell you that that is not possible and you are simply lacking a lock somewhere and just moving on like they have just shed divine light.

First of all lets get the common misconception cleared that it is impossible for a single thread to execute code in parallel and even worse that events will always be fired with a new thread affinity. This is just plain wrong!

Create a simple empty form with the following code in the shown event:

And paste the following somewhere below:

If what everybody proclaims is right this code should just lock up the UI and neatly print “A”, “B” in sequence when triggered.
This is where the bad news starts, it doesn’t, simply clicking the button a few times very easily prints “A”, “A”, “B”, “B”.
So with a few lines of code we have disproved two very stubborn myths in one blow.

Now for a somewhat more sensible approach:

This properly locks up the UI as expected (it’s example code, not release code) and can even be used for some more interesting purposes when multi threading.

Please don’t be like all the other assholes online who proclaim that this doesn’t happen in normal production code (or even worse that a simple lock() should always be sufficient as events have different thread affinities).
Simply imagine a simple asynchronous communication protocol between two programs or a server and a client, having simple locks can seriously fuck up your day with some long debugging ahead.

MariaDB + Debian

For MariaDB fans there are some changes coming in the current Debian testing/unstable, which will hit mainstream sometime this year (I guess).
A lot of things compiled against libmariadbclient.so.18 will start failing, most of which can be solved with a simple symlink except a few…

Opendmarc wil fail to compile:
opendkim: malloc.c:3757: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned long) (nb)' failed.

Spamassassin will fail to run
spamd[27186]: at /usr/share/perl5/Mail/SpamAssassin/Conf/SQL.pm line 138.
spamd[27186]: spamd: service unavailable: Error fetching user preferences via SQL
spamd[3690]: config: failed to load user (dmarc@black-mail.nl) scores from SQL database: install_driver(mysql) failed: Attempt to reload DBD/mysql.pm aborted.
spamd[3690]: Compilation failed in require at (eval 1409) line 3, line 2.

The opendkim one is pretty vague and seemingly unhelpfull but a bit of digging will reveal the following:
Can't load '/usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so' for module DBD::mysql: /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so: symbol mysql_options4, version libmariadbclient_18 not defined in file libmariadbclient.so.18 with link time reference at /usr/lib/x86_64-linux-gnu/perl/5.24/DynaLoader.pm line 187.
at /usr/sbin/opendmarc-import line 22.
Compilation failed in require at /usr/sbin/opendmarc-import line 22.

Hey, now we have a finger to point, Perl::DBD:
ldd /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so
/usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so: /lib/libmariadbclient.so.18: no version information available (required by /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so)
linux-vdso.so.1 (0x00007ffdca9bc000)
libmariadbclient.so.18 => /lib/libmariadbclient.so.18 (0x00007faf4fe63000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faf4fac5000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007faf4f8ab000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faf4f68e000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faf4f48a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007faf4f184000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007faf4ef18000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007faf4ea85000)
/lib64/ld-linux-x86-64.so.2 (0x000056045c708000)

For some reason no symlinking will solve this, but no worries, simply get the source, unpack it and run:
perl Makefile.PL
make
make install
make clean

After this things should work smoothly, but keep in mind to keep the package up date date and remove the manually installed files should Debian fix this.

Corrupting bash on Windows

Let’s asume bash is allready installed:
sudo apt-get install bb

If you are root exit to the normal user, bb won’t launch as root.

bb -loop -dim -bold -reverse -normal -boldfont -extended -inverse

After a while (especially during the heavier animations) the line width wil vary and result in some weird looking stuff.

Effective dovecot+sieve extension filtering

After some recent conversations with a colleague about creating a simple shield that would stop some of the dangerous crypto/ransomeware I set about creating such a filter.

The premise was simple, move all mail containing predefined attachments under a certain size to a different folder. Sounds like a fine job for the sieve filter to me.

A quick search on the internet resulted in some basic ways to achieve this sort of behavior, however most were plain wrong in the execution of the regex statements. Which as it turns out should not even be used, the pigeonhole addon does all the required mime handling for us.

Problems with most solutions:

  • Regex statement per extension which creates an unreadable mess (and plain kills the server performance on larger emails)
  • Not limiting the search areas, this results in a lot of false positives (e.g. any of the extensions appearing AFTER the filename has been specified (within the encoded attachment) would trigger the rule)
  • Failing to correctly identify and target the different filename formats (filename=”test.zip”, filename=test.zip[\r\n] or filename=test.zip; and in extreme cases just a file= with no filename=)

The result code:

Now this doesn’t solve all problems presented with the current ransomeware (or other scams/virusses), but it does prevent a lot of the current exploits which were comming in in bulk.

Don’t forget to specify the folder in doveoct as a required folder:

Systemd @reboot cron replacement

Some of us enjoyed the old @reboot crontab entries but the recent upgrades to systemd have created some new challenges.
The most prominent being that a simple systemd service kills any spawned processes after it exits. (I know it is not supposed to be used like this, but for some simple services like mailgraph writing service files for everyone is just too cumbersome).

[Unit]
Description=Boot services

[Service]
Type=oneshot
RemainAfterExit=yes
KillMode=none
ExecStart=<boot-script>

DotNet simple and effective profiling

There is one thing all programmers experience at one point or another, performance issues… One might use simple break points to get a coarse idea of the problem locations or when things get really frustrating a stopwatch with several measurement points.
But even that gets frustrating when there are lots of problem areas and it soon turns into a cumbersome practice which we all loathe.

But nothing beats some good old fashioned scripting to create a more elegant solution:

Simple invoke:

More complex invoke:

Next up the extension method, sometimes it’s better to itterate a function a “few” times in order to get a better idea:

Simple invoke:

More complex invoke:

Although it won’t provide you with some of the nice graphs, heatmaps or other goodies some external applications might offer, it will save a lot of time and annoyance.

December in the it department

While December for most people is a cheerful and happy month it is also has another side most commonly experienced by the IT department.

  • Spam rises notably
  • Hack attempts rise significantly (strangely the week after the Paris attacks they skyrocketed from French ip’s)
  • Server loads become more spastic, especially in retail

This year santa claus appears to be handing out free .biz extensions as a lot of spam (neatly formed and a great pain for the spam filters I might add) is arriving from freshly registered domains.

A quick example over the last 24 hours, for every ham e-mail there are 1.6 spam! (usually it’s below a 1:1 ratio)
spam 09-12-2015
Ham 09-12-2015

PNGCrush all

Some programs are great to use but are annoying when it comes to bulk operations they are just too cumbersome, PNGCrush is one of them.
Don’t get me wrong, I really like this program (and actually use it quite often), but it needs a bit of spice to be more useful in our busy daily lives. I don’t want to figure out the syntax every usage and windows “scripting” is just to unreliable.

DotNet to the rescue!, if there is one thing this framework is useful for it’s fast and reliable programming, so a simple CLI helper is quickly formed.

  • Can easily be adapted for other programs/use cases
  • Launches one PNCrush instance per core
  • Displays files that will be processed
  • Displays neatly formatted results
  • Finds the best available PNGCrush executable (version and architecture)
  • Works with DotNet 4.5.2 and newer, older versions require some tinkering (although I don’t see valid reasons to remain on any older version)
  • Could be converted to .AsParallel() LinQ, but where is the fun in that?

PNGCrush-All