canonical_header_value: convert input strings to unicode

"from __future__ import unicode_literals" causes string literals
like r'\s+' and ' ' to be unicode, so when an incoming value is
a byte string with non-ASCII, or it is a unicode with non-ASCII
(which ends up encoded with an unspecified encoding), you get a
UnicodeDecodeError in re.sub.

Instead, decode any byte string; since there's no right choice,
but we don't want anything to throw, decode as UTF-8 but use 'replace'
so that encoding errors are not fatal.

Closes: #53

CHANGES.rst

@@ -2,11 +2,17 @@ ChangeLog
 =========
 
 
-`2.1.0 (2017-01-13) <https://github.com/kdeldycke/maildir-deduplicate/compare/v2.0.1...v2.1.0>`_
-------------------------------------------------------------------------------------------------
+`2.1.1 (unreleased) <https://github.com/kdeldycke/maildir-deduplicate/compare/v2.1.0...develop>`_
+-------------------------------------------------------------------------------------------------
 
 .. note:: This version is not yet released and is under active development.
 
+* No changes yet.
+
+
+`2.1.0 (2017-01-13) <https://github.com/kdeldycke/maildir-deduplicate/compare/v2.0.1...v2.1.0>`_
+------------------------------------------------------------------------------------------------
+
 * Fix rendering of changelog link in RST.
 * Show selected log level in debug mode.
 * Test builds against Python 3.6 and 3.7-dev, and most recent PyPy targetting

README.rst

@@ -42,3 +42,22 @@ Features
 * Dry-run mode.
 * Protection against false-positives by checking for size and content
   differences.
+
+
+Installation
+------------
+
+This package is `available on PyPi
+<https://pypi.python.org/pypi/maildir-deduplicate>`_, so you can install the
+latest stable release and its dependencies with a simple ``pip`` call:
+
+.. code-block:: shell-session
+
+    $ pip install maildir-deduplicate
+
+
+Documentation
+-------------
+
+Docs are `hosted on Read the Docs
+<https://maildir-deduplicate.readthedocs.io>`_.

maildir_deduplicate/__init__.py

@@ -24,7 +24,7 @@ import logging
 import sys
 
 
-__version__ = '2.1.0'
+__version__ = '2.1.1'
 
 
 PY2 = sys.version_info[0] == 2

maildir_deduplicate/mail.py

@@ -182,8 +182,8 @@ class Mail(object):
         header = header.lower()
         # Problematic when reading utf8 emails
         # this will ensure value is always string
-        if not type(value) is str:
-            value = value.encode()
+        if isinstance(value, bytes):
+            value = value.decode('utf-8', 'replace')
         value = re.sub(r'\s+', ' ', value).strip()
 
         # Trim Subject prefixes automatically added by mailing list software,

setup.cfg

@@ -6,7 +6,7 @@ universal = 1
 
 # https://github.com/peritus/bumpversion#configuration
 [bumpversion]
-current_version = 2.1.0
+current_version = 2.1.1
 files = ./maildir_deduplicate/__init__.py ./CHANGES.rst
 allow_dirty = True
 commit = False