Simple yet flexible natural sorting in Python.
natsort is a general utility for sorting lists naturally; the
definition of “naturally” is not well-defined, but the most common definition
is that numbers contained within the string should be sorted as numbers and not
as you would other characters. If you need to present sorted output to a user,
you probably want to sort it naturally.
natsort was initially created for sorting scientific output filenames that
contained signed floating point numbers in the names. There was a lack of
algorithms out there that could perform a natural sort on floats but
plenty for ints; check out
this StackOverflow question
and its answers and links therein,
this ActiveState forum,
and of course this great article on natural sorting
from CodingHorror.com for examples of what I mean.
natsort was created to fill in this gap, but has since expanded to
handle just about any definition of a number, as well as other sorting
1.1. Quick Description¶
When you try to sort a list of strings that contain numbers, the normal python sort algorithm sorts lexicographically, so you might not get the results that you expect:
>>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in'] >>> sorted(a) ['1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '2 ft 7 in', '7 ft 6 in']
Notice that it has the order (‘1’, ‘10’, ‘2’) - this is because the list is being sorted in lexicographical order, which sorts numbers like you would letters (i.e. ‘b’, ‘ba’, ‘c’).
natsort provides a function
natsorted() that helps sort lists
“naturally” (“naturally” is rather ill-defined, but in general it means
sorting based on meaning and not computer code point)..
natsorted() is simple:
>>> from natsort import natsorted >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in'] >>> natsorted(a) ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']
natsorted() identifies numbers anywhere in a string and sorts them
naturally. Below are some other things you can do with
(please see the Examples and Recipes for a quick start guide, or the natsort API
for more details).
natsorted() is designed to be a drop-in replacement for the built-in
sorted() function. Like
does not sort in-place. To sort a list and assign the output to the
same variable, you must explicitly assign the output to a variable:
>>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in'] >>> natsorted(a) ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in'] >>> print(a) # 'a' was not sorted; "natsorted" simply returned a sorted list ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in'] >>> a = natsorted(a) # Now 'a' will be sorted because the sorted list was assigned to 'a' >>> print(a) ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']
Please see Generating a Reusable Sorting Key and Sorting In-Place for an alternate way to sort in-place naturally.
1.2.1. Sorting Versions¶
natsort does not (and never has) actually comprehend version numbers.
It just so happens that the most common versioning schemes are designed to
work with standard natural sorting techniques; these schemes include
YEAR.MONTH.DAY. If your data
conforms to a scheme like this, then it will work out-of-the-box with
natsorted (as of
natsort version >= 4.0.0):
>>> a = ['version-1.9', 'version-2.0', 'version-1.11', 'version-1.10'] >>> natsorted(a) ['version-1.9', 'version-1.10', 'version-1.11', 'version-2.0']
If you need to versions that use a more complicated scheme, please see Sorting More Expressive Versioning Schemes for examples.
1.2.2. Sorting by Real Numbers (i.e. Signed Floats)¶
>>> from natsort import realsorted, ns >>> # Note that when interpreting as signed floats, the below numbers are >>> # +5.10, -3.00, +5.30, +2.00 >>> a = ['position5.10.data', 'position-3.data', 'position5.3.data', 'position2.data'] >>> natsorted(a) ['position2.data', 'position5.3.data', 'position5.10.data', 'position-3.data'] >>> natsorted(a, alg=ns.REAL) ['position-3.data', 'position2.data', 'position5.10.data', 'position5.3.data'] >>> realsorted(a) # shortcut for natsorted with alg=ns.REAL ['position-3.data', 'position2.data', 'position5.10.data', 'position5.3.data']
1.2.3. Locale-Aware Sorting (or “Human Sorting”)¶
This is where the non-numeric characters are ordered based on their meaning,
not on their ordinal value, and a locale-dependent thousands separator and decimal
separator is accounted for in the number.
This can be achieved with the
>>> a = ['Apple', 'apple15', 'Banana', 'apple14,689', 'banana'] >>> natsorted(a) ['Apple', 'Banana', 'apple14,689', 'apple15', 'banana'] >>> import locale >>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8') 'en_US.UTF-8' >>> natsorted(a, alg=ns.LOCALE) ['apple15', 'apple14,689', 'Apple', 'banana', 'Banana'] >>> from natsort import humansorted >>> humansorted(a) ['apple15', 'apple14,689', 'Apple', 'banana', 'Banana']
You may find you need to explicitly set the locale to get this to work
(as shown in the example).
Please see Possible Issues with humansorted() or ns.LOCALE and the Installation section
below before using the
1.2.4. Further Customizing Natsort¶
If you need to combine multiple algorithm modifiers (such as
ns.IGNORECASE), you can combine the options using the
bitwise OR operator (
|). For example,
>>> a = ['Apple', 'apple15', 'Banana', 'apple14,689', 'banana'] >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE) ['Apple', 'apple15', 'apple14,689', 'Banana', 'banana'] >>> # The ns enum provides long and short forms for each option. >>> ns.LOCALE == ns.L True >>> # You can also customize the convenience functions, too. >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE) == realsorted(a, alg=ns.L | ns.IC) True >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE) == humansorted(a, alg=ns.R | ns.IC) True
All of the available customizations can be found in the documentation for
You can also add your own custom transformation functions with the
argument. These can be used with
alg if you wish:
>>> a = ['apple2.50', '2.3apple'] >>> natsorted(a, key=lambda x: x.replace('apple', ''), alg=ns.REAL) ['2.3apple', 'apple2.50']
1.2.5. Sorting Mixed Types¶
You can mix and match
when you sort:
>>> a = ['4.5', 6, 2.0, '5', 'a'] >>> natsorted(a) [2.0, '4.5', '5', 6, 'a'] >>> # On Python 2, sorted(a) would return [2.0, 6, '4.5', '5', 'a'] >>> # On Python 3, sorted(a) would raise an "unorderable types" TypeError
1.2.6. Handling Bytes on Python 3¶
natsort does not officially support the bytes type on Python 3, but
convenience functions are provided that help you decode to str first:
>>> from natsort import as_utf8 >>> a = [b'a', 14.0, 'b'] >>> # On Python 2, natsorted(a) would would work as expected. >>> # On Python 3, natsorted(a) would raise a TypeError (bytes() < str()) >>> natsorted(a, key=as_utf8) == [14.0, b'a', 'b'] True >>> a = [b'a56', b'a5', b'a6', b'a40'] >>> # On Python 2, natsorted(a) would would work as expected. >>> # On Python 3, natsorted(a) would return the same results as sorted(a) >>> natsorted(a, key=as_utf8) == [b'a5', b'a6', b'a40', b'a56'] True
1.2.7. Generating a Reusable Sorting Key and Sorting In-Place¶
Under the hood,
natsorted() works by generating a custom sorting
natsort_keygen() and then passes that to the built-in
sorted(). You can use the
natsort_keygen() function yourself to
generate a custom sorting key to sort in-place using the
>>> from natsort import natsort_keygen >>> natsort_key = natsort_keygen() >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in'] >>> natsorted(a) == sorted(a, key=natsort_key) True >>> a.sort(key=natsort_key) >>> a ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']
1.2.8. Other Useful Things¶
- How do I debug
The best way to debug
natsorted()is to generate a key using
natsort_keygen()with the same options being passed to
natsorted(). One can take a look at exactly what is being done with their input using this key - it is highly recommended to look at this issue describing how to debug for how to debug, and also to review the How Does Natsort Work? page for why
natsortis doing that to your data.
If you are trying to sort custom classes and running into trouble, please take a look at https://github.com/SethMMorton/natsort/issues/60. In short, custom classes are not likely to be sorted correctly if one relies on the behavior of
__lt__and the other rich comparison operators in their custom class - it is better to use a
natsort, or use the
natsortkey as part of your rich comparison operator definition.
natsortgave me results I didn’t expect, and it’s a terrible library!
- Did you try to debug using the above advice? If so, and you still cannot figure out the error, then please file an issue.
- How does
If you don’t want to read How Does Natsort Work?, here is a quick primer.
natsortprovides a key function that can be passed to
sorted()in order to modify the default sorting behavior. This key is generated on-demand with the key generator
natsort.natsorted()is essentially a wrapper for the following code:
>>> from natsort import natsort_keygen >>> natsort_key = natsort_keygen() >>> sorted(['1', '10', '2'], key=natsort_key) ['1', '2', '10']
- Assume the input is a string, and attempt to split it into numbers and
non-numbers using regular expressions. Numbers are then converted into
- If the above fails because the input is not a string, assume the input
is some other sequence (e.g.
tuple), and recursively apply the key to each element of the sequence.
- If the above fails because the input is not iterable, assume the input
float, and just return the input in a
- Assume the input is a string, and attempt to split it into numbers and non-numbers using regular expressions. Numbers are then converted into either
1.4. Shell script¶
1.6. Optional Dependencies¶
The most efficient sorting can occur if you install the
(version >=2.0.0); it helps with the string to number conversions.
natsort will still run (efficiently) without the package, but if you
need to squeeze out that extra juice it is recommended you include this as a
natsort will not require (or check) that
fastnumbers is installed
$ pip install natsort
# Install both optional dependencies. $ pip install natsort[fast,icu] # Install just fastnumbers $ pip install natsort[fast]
1.8. How to Run Tests¶
Please note that
natsort is NOT set-up to support
python setup.py test.
The recommended way to run tests is with tox.
tox, running tests is as simple as executing the following
tox will create virtual a virtual environment for your tests and install
all the needed testing requirements for you. You can specify a particular
python version with the
-e flag, e.g.
tox -e py36. Static analysis is
tox -e flake8. You can see all available testing environments
If you do not wish to use
tox, you can install the testing dependencies with the
dev/requirements.txt file and then run the tests manually using
$ pip install -r dev/requirements.txt $ python -m pytest
Note that above I invoked
python -m pytest instead of just
pytest - this is because
the former puts the CWD on sys.path.
1.9. How to Build Documentation¶
If you want to build the documentation for
natsort, it is recommended to
$ tox -e docs
This will place the documentation in
build/sphinx/html. If you do not
which to use
tox, you can do the following:
$ pip install sphinx sphinx_rtd_theme $ python setup.py build_sphinx
1.10. Deprecation Schedule¶
1.10.1. Dropping Python 2.7 Support¶
natsort version 7.0.0 will drop support for Python 2.7.
The version 6.X branch will remain as a “long term support” branch where bug
fixes are applied so that users who cannot update from Python 2.7 will not be
forced to use a buggy
natsort version. Once version 7.0.0 is released,
new features will not be added to version 6.X, only bug fixes.
1.10.2. Deprecated APIs¶
natsort version 6.0.0, the following APIs and functions were removed
number_typekeyword argument (deprecated since 3.4.0)
signedkeyword argument (deprecated since 3.4.0)
expkeyword argument (deprecated since 3.4.0)
as_pathkeyword argument (deprecated since 3.4.0)
py3_safekeyword argument (deprecated since 3.4.0)
ns.TYPESAFE(deprecated since version 5.0.0)
ns.DIGIT(deprecated since version 5.0.0)
ns.VERSION(deprecated since version 5.0.0)
versorted()(discouraged since version 4.0.0, officially deprecated since version 5.5.0)
index_versorted()(discouraged since version 4.0.0, officially deprecated since version 5.5.0)
In general, if you want to determine if you are using deprecated APIs you can run your code with the following flag
$ python -Wdefault::DeprecationWarning my-code.py
DeprecationWarnings are not shown, but this will cause them
to be shown. Alternatively, you can just set the environment variable
PYTHONWARNINGS to “default::DeprecationWarning” and then run your code.