C# things that make us happy

Sometimes our lives as programmers get a bit easier, Microsoft has launchedlaunched C# 7.0 some time ago. Here are a few examples of the things that made our lives a lot easier.

The microsoft page has better explanations, the code below is just random code from my project. These examples are only intended to outline some of the new features.

Pattern matching

            if (sender is ToolStripMenuItemTranslate)
            {
                ToolStripMenuItemTranslate item = (ToolStripMenuItemTranslate)sender;
                TabPageAddOrShow(item.CustomTabPage, ModifierKeys == Keys.Shift);
            }

Can be rewriten as:

            if (sender is ToolStripMenuItemTranslate item)
            {
                TabPageAddOrShow(item.CustomTabPage, ModifierKeys == Keys.Shift);
            }

Using the underscore we can ignore variables, which keeps the compiler (and resharper) happy:
DoSomething(out String variable1, out String _)
Which brings us the pleasure of inline variable definition as well as ignoring unused ones.

It can also create funny code (introducing underscores in useless places):

            if (sender is ToolStripMenuItemTranslate _)

Tuples
While we have had basic tuples for a long time they grew up and gave us a weapon against the useless class/structs to perform returns (or way to many out variables).

        public (String results, Int32 exitCode) ExecuteCommand(String command)

Keep in mind that a tuple is a struct!

Switch statements with patterns
While switching still isn’t available on primitive types it sure improves our lives a lot:

                switch (Columns[i])
                {
                    case DataGridViewTextBoxColumnDatabaseBackendTranslate a:
                        if (!a.CustomOnlyOnceCombined) continue;

                        columns.Add(i);
                        break;
                    case DataGridViewTextBoxColumnCustomIDTranslate b:
                        if (!b.CustomOnlyOnceCombined) continue;

                        columns.Add(i);
                        break;
                    case DataGridViewComboBoxColumnArrayBackendAndTranslate c:
                        if (!c.CustomOnlyOnceCombined) continue;

                        columns.Add(i);
                        break;
                    case DataGridViewComboBoxColumnDatabaseBackendAndTranslate d:
                        if (!d.CustomOnlyOnceCombined) continue;

                        columns.Add(i);
                        break;
                }

The nature of C# locking issues explained

Sometimes in the life of a programmer everything seems peachy, C#/DotNet provide relatively easy multi threading capabilities which 99.9% of the time work just fine as explained everywhere online.

However when things get a bit serious as these online explanations tend to be off by a small bit, only that small bit creates havoc….

Imagine having to debug code within a lock() statement that crashes with exceptions like “Collection was modified; enumeration operation may not execute” which seemingly bypass any locking in place.
These are not fun to deal with as everyone will for some reason stubbornly tell you that that is not possible and you are simply lacking a lock somewhere and just moving on like they have just shed divine light.

First of all lets get the common misconception cleared that it is impossible for a single thread to execute code in parallel and even worse that events will always be fired with a new thread affinity. This is just plain wrong!

Create a simple empty form with the following code in the shown event:

            Button testButton = new Button { Dock = DockStyle.Fill };
            testButton.Click += Test_Click;
            Controls.Add(testButton);

And paste the following somewhere below:

        private readonly object _lockHelper = new object();
        private void Test_Click(object sender, EventArgs e)
        {
            lock(_lockHelper)
            {
                Debug.WriteLine("A");

                int i = 0;
                while (i++ < 50)
                {
                    Application.DoEvents();
                    Thread.Sleep(50);
                }

                Debug.WriteLine("B");
            }
        }

If what everybody proclaims is right this code should just lock up the UI and neatly print "A", "B" in sequence when triggered.
This is where the bad news starts, it doesn't, simply clicking the button a few times very easily prints "A", "A", "B", "B".
So with a few lines of code we have disproved two very stubborn myths in one blow.

Now for a somewhat more sensible approach:

        private static readonly SemaphoreSlim _lockHelper = new SemaphoreSlim(1, 1);
        private void Test_Click(object sender, EventArgs e)
        {
            _lockHelper.Wait();
            Debug.WriteLine("A");

            int i = 0;
            while (i++ < 50)
            {
                Application.DoEvents();
                Thread.Sleep(50);
            }

            Debug.WriteLine("B");
            _lockHelper.Release();
        }

This properly locks up the UI as expected (it's example code, not release code) and can even be used for some more interesting purposes when multi threading.

Please don't be like all the other assholes online who proclaim that this doesn't happen in normal production code (or even worse that a simple lock() should always be sufficient as events have different thread affinities).
Simply imagine a simple asynchronous communication protocol between two programs or a server and a client, having simple locks can seriously fuck up your day with some long debugging ahead.

DotNet simple and effective profiling

There is one thing all programmers experience at one point or another, performance issues… One might use simple break points to get a coarse idea of the problem locations or when things get really frustrating a stopwatch with several measurement points.
But even that gets frustrating when there are lots of problem areas and it soon turns into a cumbersome practice which we all loathe.

But nothing beats some good old fashioned scripting to create a more elegant solution:

    public class SimpleProfiler : IDisposable
    {
        private readonly Stopwatch _stopWatch = new Stopwatch();
        private readonly int _measurementFormatLength;
        private readonly bool _printFinalResult;
        public SimpleProfiler(string title = null, int measurementFormatLength = 30, bool printFinalResult = true)
        {
            _measurementFormatLength = measurementFormatLength;
            _printFinalResult = printFinalResult;
            if (title != null)
            {
                Debug.WriteLine(title);
            }
            _stopWatch.Start();
        }
        public void Restart()
        {
            _stopWatch.Restart();
        }
        public void Dispose()
        {
            _stopWatch.Stop();
            if (_printFinalResult)
            {
                Debug.WriteLine("Elapsed {0} ms", _stopWatch.ElapsedMilliseconds);
            }
        }

        public void AddMeasurement(string comment = "")
        {
            Debug.WriteLine(" {0} {1} ms", comment.PadRight(_measurementFormatLength).Substring(0, _measurementFormatLength), _stopWatch.ElapsedMilliseconds);
        }
    }

Simple invoke:

            using (new SimpleProfiler())
            {
                WebClient webClient = new WebClient();
                webClient.DownloadString("http://www.black-mail.nl/");
            }

More complex invoke:

            using (SimpleProfiler profiler = new SimpleProfiler("Download test", 15, false))
            {
                WebClient webClient = new WebClient();
                webClient.DownloadString("http://www.black-mail.nl/");
                profiler.AddMeasurement("http test");
                profiler.Restart();
                webClient.DownloadString("https://www.black-mail.nl/");
                profiler.AddMeasurement("https test");
            }

Next up the extension method, sometimes it’s better to itterate a function a “few” times in order to get a better idea:

    public static class ExtensionMethods
    {
        public static Int64 Profile(this T input, Action action, int iterations = 1)
        {
            Stopwatch stopwatch = new Stopwatch();
            stopwatch.Start();
            for (int i = 0; i < iterations; i++)
            {
                action(input);
            }
            stopwatch.Stop();

            return stopwatch.ElapsedMilliseconds;
        }

        public static Int64 Profile(this Action action, int iterations = 1)
        {
            Stopwatch stopwatch = new Stopwatch();
            stopwatch.Start();
            for (int i = 0; i < iterations; i++)
            {
                action();
            }
            stopwatch.Stop();

            return stopwatch.ElapsedMilliseconds;
        }
    }

Simple invoke:

            Action action = LongCalculation;
            Debug.WriteLine(action.Profile().ToString(CultureInfo.InvariantCulture));

More complex invoke:

            Action action = LongCalculation;
            Debug.WriteLine(action.Profile(x => x.Invoke("Some input")).ToString(CultureInfo.InvariantCulture));

Although it won't provide you with some of the nice graphs, heatmaps or other goodies some external applications might offer, it will save a lot of time and annoyance.

PNGCrush all

Some programs are great to use but are annoying when it comes to bulk operations they are just too cumbersome, PNGCrush is one of them.
Don’t get me wrong, I really like this program (and actually use it quite often), but it needs a bit of spice to be more useful in our busy daily lives. I don’t want to figure out the syntax every usage and windows “scripting” is just to unreliable.

DotNet to the rescue!, if there is one thing this framework is useful for it’s fast and reliable programming, so a simple CLI helper is quickly formed.

  • Can easily be adapted for other programs/use cases
  • Launches one PNCrush instance per core
  • Displays files that will be processed
  • Displays neatly formatted results
  • Finds the best available PNGCrush executable (version and architecture)
  • Works with DotNet 4.5.2 and newer, older versions require some tinkering (although I don’t see valid reasons to remain on any older version)
  • Could be converted to .AsParallel() LinQ, but where is the fun in that?
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

namespace CrushAll
{
    public class Program
    {
        public static void Main(string[] args)
        {
            _args = args;
            try
            {
                CrushAll();
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception.ToString());
            }

            Console.WriteLine("Done, press any key to exit");
            Console.ReadLine();
        }

        private static string[] _args;
        private static readonly List ActiveTasks = new List();
        private static string _pngCrush = "";
        private static string _currentPath = "";
        private static void CrushAll()
        {
            _currentPath = AppDomain.CurrentDomain.BaseDirectory;
            List files = ReturnFiles(_currentPath, true, new List { ".png" }).ToList();
            files.AddRange(_args.Where(argument => argument.EndsWith(".png", StringComparison.OrdinalIgnoreCase) && File.Exists(argument)));
            IEnumerable executables = ReturnFiles(_currentPath, true, new List { ".exe" });

            // ReSharper disable PossibleMultipleEnumeration
            if (!files.Any())
            {
                Console.WriteLine("No images found!");
                return;
            }
            if (!executables.Any())
            {
                Console.WriteLine("No PNG Crush executable(s) found!");
                return;
            }

            List possibleExecutables = new List();
            foreach (string file in executables.Select(Path.GetFileName).Where(file => file.StartsWith("pngcrush_", StringComparison.OrdinalIgnoreCase)))
            {
                try
                {
                    possibleExecutables.Add(new PossibleExecutable { X64 = file.Contains("_w64"), Version = Version.Parse(file.Substring(file.IndexOf("_", StringComparison.Ordinal) + 1, file.LastIndexOf("_", StringComparison.Ordinal) - (file.IndexOf("_", StringComparison.Ordinal) + 1)).Replace("_", ".")), Executable = file});
                }
                catch (Exception) {}
            }
            // ReSharper restore PossibleMultipleEnumeration

            Version highestX64 = new Version(0, 0);
            Version highestX86 = new Version(0, 0);
            foreach (PossibleExecutable executable in possibleExecutables)
            {
                if (executable.Version > highestX86 && !executable.X64)
                {
                    highestX86 = executable.Version;
                }
                if (executable.Version > highestX64 && executable.X64)
                {
                    highestX64 = executable.Version;
                }
            }
            
            if (highestX86 > highestX64 && Environment.Is64BitOperatingSystem)
            {
                foreach (PossibleExecutable executable in possibleExecutables.Where(executable => executable.Version == highestX86))
                {
                    _pngCrush = executable.Executable;
                    break;
                }
            }
            else if (highestX64.Major != 0 && Environment.Is64BitOperatingSystem)
            {
                foreach (PossibleExecutable executable in possibleExecutables.Where(executable => executable.Version == highestX64 && executable.X64))
                {
                    _pngCrush = executable.Executable;
                    break;
                }
            }
            else if (highestX86.Major != 0)
            {
                foreach (PossibleExecutable executable in possibleExecutables.Where(executable => executable.Version == highestX86 && !executable.X64))
                {
                    _pngCrush = executable.Executable;
                    break;
                }
            }

            if (string.IsNullOrEmpty(_pngCrush))
            {
                Console.WriteLine("No suitable executable found!");
                return;
            }

            Console.WriteLine("Preparing to crush:");
            Console.WriteLine("Thread(s): " + Environment.ProcessorCount.ToString(CultureInfo.InvariantCulture));
            Console.WriteLine("Executable: " + _pngCrush.Replace(_currentPath, ""));

            Queue queue = new Queue();
            foreach (string file in files)
            {
                queue.Enqueue(file);
                Console.WriteLine("File: " + file.Replace(_currentPath, ""));
            }
            
            while (queue.Any())
            {
                Thread.Sleep(500);

                lock (ActiveTasks)
                {
                    while (queue.Any())
                    {
                        if (Environment.ProcessorCount <= ActiveTasks.Count) break;
                        string file = queue.Dequeue();
                        Task task = Task.Factory.StartNew(() => { ProcessFile(file); }, TaskCreationOptions.LongRunning);
                        task.ContinueWith(t => HandleException(t.Exception), TaskContinuationOptions.OnlyOnFaulted);
                        task.ContinueWith(DoneProcessing, TaskContinuationOptions.None);
                        ActiveTasks.Add(task);
                    }
                }
            }

            while (true)
            {
                lock (ActiveTasks)
                {
                    Thread.Sleep(500);
                    if (!ActiveTasks.Any())
                    {
                        break;
                    }
                }
            }
        }
        private static void HandleException(AggregateException exception)
        {
            foreach (Exception ex in exception.Flatten().InnerExceptions)
            {
                Console.WriteLine(ex.ToString());
            }
        }
        private static void DoneProcessing(Task task)
        {
            lock (ActiveTasks)
            {
                ActiveTasks.Remove(task);
            }
        }

        public static void ProcessFile(string file)
        {
            string tempFile = file + ".tmp";
            Int64 orriginalLength, newLength;
            using (FileStream fileInfo = File.Open(file, FileMode.Open, FileAccess.Read, FileShare.Write))
            {
                orriginalLength = fileInfo.Length;
            }
            
            // ReSharper disable once AssignNullToNotNullAttribute
            ProcessStartInfo processStartInfo = new ProcessStartInfo { FileName = _pngCrush, Arguments = "-brute -q -reduce \"" + file + "\" \"" + file + ".tmp\"", WorkingDirectory = Path.GetDirectoryName(_pngCrush), CreateNoWindow = true, UseShellExecute = false };
            Process process = Process.Start(processStartInfo);
            if (process != null)
            {
                process.WaitForExit();
                process.Close();
                process.Dispose();
            }
            else
            {
                Console.WriteLine("Could not start PNGCrush!");
                return;
            }

            if (!File.Exists(tempFile))
            {
                Console.WriteLine("Could not recompress: " + file.Replace(_currentPath, ""));
                return;
            }

            using (FileStream fileInfo = File.Open(tempFile, FileMode.Open, FileAccess.Read, FileShare.Write))
            {
                newLength = fileInfo.Length;
            }
            
            if (newLength > orriginalLength)
            {
                Console.WriteLine(file.Replace(_currentPath, "") + ": resize was bigger, ignoring");
            }
            else if (newLength == orriginalLength)
            {
                Console.WriteLine(file.Replace(_currentPath, "") + ": resize was equal, ignoring");
            }
            else
            {
                Console.WriteLine(file.Replace(_currentPath, "") + ": reduced by " + FormatBytes(orriginalLength - newLength));

                int retries = 0;

                // File operations can be a bit quirky in DotNet, sometimes the file handle isn't released on time so we wait a bit and retry
retry:
                File.Delete(file);
                if (File.Exists(file))
                {
                    if (retries > 3)
                    {
                        Console.WriteLine(file.Replace(_currentPath, "") + ": could not be deleted");
                        return;
                    }
                    Thread.Sleep(250);
                    retries++;
                    goto retry;
                }

                retries = 0;

retry2:
                File.Move(tempFile, file);
                if (!File.Exists(file))
                {
                    if (retries > 3)
                    {
                        Console.WriteLine(file.Replace(_currentPath, "") + ": could not recreated");
                        return;
                    }
                    Thread.Sleep(250);
                    retries++;
                    goto retry2;
                }
            }

            // Don't remove the tempFile if the original no longer exists
            if (File.Exists(tempFile) && File.Exists(file))
            {
                File.Delete(tempFile);
            }
        }
        
        public static IEnumerable ReturnFiles(string localPath, bool recurseDirectories, List includedExtensions = null)
        {
            if (!Directory.Exists(localPath)) return new List();

            try
            {
                List directories = new List();
                if (recurseDirectories)
                {
                    directories.AddRange(RecurseDirectories(localPath).ToArray());
                }
                else
                {
                    directories.Add(localPath);
                }

                List list = new List();
                foreach (string directory in directories)
                {
                    DirectoryInfo directoryInfo = new DirectoryInfo(directory);
                    list.AddRange(from file in directoryInfo.GetFiles() where !(includedExtensions != null && !includedExtensions.Contains(file.Extension, StringComparer.OrdinalIgnoreCase)) select directory + file);
                }
                return list;
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception.ToString());
                return new List();
            }
        }
        public static IEnumerable RecurseDirectories(string localPath)
        {
            List directoriesReturn = new List();
            try
            {
                string[] directories = Directory.GetDirectories(localPath, "*", SearchOption.AllDirectories);
                directoriesReturn.AddRange(directories.Reverse());
                directoriesReturn.Add(localPath);
            }
            catch (Exception exception)
            {
                Console.WriteLine(exception.ToString());
            }

            return directoriesReturn;
        }

        private static readonly string[] Sizes = { "B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB" };
        public static string FormatBytes(Int64 bytes)
        {
            int order = 0;
            double size = bytes;
            while (size >= 1024 && order + 1 < Sizes.Length)
            {
                order++;
                size = size / 1024;
            }
            return $"{size:0.##}{Sizes[order]}";
        }
    }

    public class PossibleExecutable
    {
        public string Executable;
        public bool X64;
        public Version Version;
    }
}

PNGCrush-All

Calling Synchronous Methods Asynchronously

Using the BeginInvoke function is way to much fun and something that entry level C# programmers get their hands on fairly quickly. That doesn’t mean it doesn’t have its pitfalls as most don’t bother to read the documentation.

The Microsoft article clearly states:

  • No matter which technique you use, always call EndInvoke to complete your asynchronous call
  • EndInvoke blocks the calling thread until it completes

In situations where a blocking call is not a problem it is a quick and easy fix, but in most situations it just becomes tedious to clean up after yourself.

        public static void AutoEndInvoke(this IAsyncResult result, ISynchronizeInvoke control)
        {
            Task task = Task.Factory.StartNew(() => { AutoEndInvokeTask(result, control); }, TaskCreationOptions.PreferFairness);
            task.ContinueWith(t => HandleException(t.Exception), TaskContinuationOptions.OnlyOnFaulted);
        }
        private static void AutoEndInvokeTask(IAsyncResult result, ISynchronizeInvoke control)
        {
            control.EndInvoke(result);
        }

Using the code is easy to:

            _window.BeginInvoke(new Action(OpenFileWindow)).AutoEndInvoke(_window);

It isn’t the cleanest of solutions, it spoils a whole thread so it might not be the best solution for very heavy situations (bulk loading invokes works way better in those cases). But it does alleviate most of the occasional BeginInvoke calls.