Building Pandas from source: conflicting types for ‘_xgetbv’ (while building BLOSC)

Quick note: I tried building pandas from source (Fedora 30).

During

$ python -m pip install -r requirements-dev.txt

I ran into

blosc/shuffle.c:177:1: error: conflicting types for ‘_xgetbv’

A little more context:

$ python -m pip install -r requirements-dev.txt
 
[...]
 
Installing collected packages: blosc, bottleneck, numexpr, soupsieve, beautifulsoup4, llvmlite, numba, thrift, fastparquet, html5lib, lxml, jdcal, et-xmlfile, openpyxl, pyarrow, PyQt5-sip, pyqt5, tables, python-snappy, s3fs, sqlalchemy, xarray, xlrd, xlsxwriter, xlwt, odfpy, pyreadstat, pandas-sphinx-theme
  Running setup.py install for blosc ... error
    Complete output from command /home/jp/.pyenv/versions/374-pandas-dev/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-fydbjk8m/blosc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-x9bwxqkv/install-record.txt --single-version-externally-managed --compile --install-headers /home/jp/.pyenv/versions/374-pandas-dev/include/site/python3.7/blosc:
    SSE2 detected
    AVX2 detected
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/blosc
 
[...]
 
    c-blosc/blosc/shuffle.c:177:1: error: conflicting types for ‘_xgetbv’
      177 | _xgetbv(uint32_t xcr) {
          | ^~~~~~~
    In file included from /usr/lib/gcc/x86_64-redhat-linux/9/include/immintrin.h:43,
                     from c-blosc/blosc/blosc-common.h:71,
                     from c-blosc/blosc/shuffle.h:18,
                     from c-blosc/blosc/shuffle.c:10:
    /usr/lib/gcc/x86_64-redhat-linux/9/include/xsaveintrin.h:60:1: note: previous definition of ‘_xgetbv’ was here
       60 | _xgetbv (unsigned int __A)
          | ^~~~~~~
    In file included from c-blosc/blosc/shuffle.c:11:
    c-blosc/blosc/shuffle-generic.h:61:13: warning: ‘unshuffle_generic_inline’ defined but not used [-Wunused-function]
       61 | static void unshuffle_generic_inline(const size_t type_size,
          |             ^~~~~~~~~~~~~~~~~~~~~~~~
    c-blosc/blosc/shuffle-generic.h:32:13: warning: ‘shuffle_generic_inline’ defined but not used [-Wunused-function]
       32 | static void shuffle_generic_inline(const size_t type_size,
          |             ^~~~~~~~~~~~~~~~~~~~~~
    error: command 'gcc' failed with exit status 1

The python-blosc documentation says :

Compiler specific optimisations are automatically enabled by inspecting the CPU flags building Blosc. They can be manually disabled by setting the following environmental variables: DISABLE_BLOSC_SSE2 and DISABLE_BLOSC_AVX2.

I ignorantly retried building with AVX2 instructions disabled:

$ DISABLE_BLOSC_AVX2=true pip install blosc
Collecting blosc
  Using cached https://files.pythonhosted.org/packages/6d/3b/2b707cd330a205ba5c69b5e8bfa9c05691442e45ce9ce882c4c8d343e61a/blosc-1.8.1.tar.gz
Installing collected packages: blosc
  Running setup.py install for blosc ... done
Successfully installed blosc-1.8.1
$ python -m pip install -r requirements-dev.txt
...
$ python setup.py build_ext --inplace -j 4
...

That worked. In my development setup I do not care about BLOSC performance, which is why I am OK with that workaround.

By the way, I needed to set up the following dependencies on Fedora 30:

sudo dnf install @development-tools
sudo dnf install libzstd-devel gcc-c++ snappy-devel

Leave a Reply to Jan-Philip Gehrcke Cancel reply

Your email address will not be published. Required fields are marked *

Human? Please fill this out: * Time limit is exhausted. Please reload CAPTCHA.

  1. Chris Avatar
    Chris

    Alas, I’ve got the same issue, but disabling the CPU flags (both), didn’t help. Did you go down any other rabbit holes?

    1. Jan-Philip Gehrcke Avatar

      Mhm, no sorry, I have no further pointers w/o looking deeply again. If you find something please let us know! Good luck..