在python2中加载python3训练和保存的模型时出错:
ValueErrorTraceback (most recent call last)
--> 237 clf = joblib.load('clf300_all.model')
238 pred_y = clf.predict_proba(X)
/usr/local/anaconda2/lib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle.pyc in load(filename, mmap_mode)
576 return load_compatibility(fobj)
577
--> 578 obj = _unpickle(fobj, filename, mmap_mode)
579
580 return obj
/usr/local/anaconda2/lib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)
ValueError: unsupported pickle protocol: 3
经过查阅资料:
跨python版本的 joblib.dump() 和 joblib.load()
Compatibility across python versions
Compatibility of joblib pickles across python versions is not fully supported. Note that, for a very restricted set of objects, this may appear to work when saving a pickle with python 2 and loading it with python 3 but relying on it is strongly discouraged.
If you are switching between python versions, you will need to save a different joblib pickle for each python version.
Here are a few examples or exceptions:
-
Saving joblib pickle with python 2, trying to load it with python 3:
Traceback (most recent call last):
File "/home/lesteve/dev/joblib/joblib/numpy_pickle.py", line 453, in load
obj = unpickler.load()
File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1038, in load
dispatch[key[0]](self)
File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1176, in load_binstring
self.append(self._decode_string(data))
File "/home/lesteve/miniconda3/lib/python3.4/pickle.py", line 1158, in _decode_string
return value.decode(self.encoding, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 1024: ordinal not in range(128)
Traceback (most recent call last):
File "", line 1, in <module>
File "/home/lesteve/dev/joblib/joblib/numpy_pickle.py", line 462, in load
raise new_exc
ValueError: You may be trying to read with python 3 a joblib pickle generated with python 2. This is not feature supported by joblib.
-
Saving joblib pickle with python 3, trying to load it with python 2:
Traceback (most recent call last):
File "", line 1, in <module>
File "joblib/numpy_pickle.py", line 453, in load
obj = unpickler.load()
File "/home/lesteve/miniconda3/envs/py27/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/home/lesteve/miniconda3/envs/py27/lib/python2.7/pickle.py", line 886, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3
=================================================================================================================================================
不完全支持跨python版本的joblib pickle的兼容性。请注意,对于一组非常有限的对象,当使用python 2保存pickle并使用python 3加载它时,这可能会起作用,但强烈建议不要依赖它。
如果要在python版本之间切换,则需要为每个python版本保存不同的joblib pickle。
==================================================================================================================================================
另外:不同python版本的pickle.dump()和pickle.load()是可以相互转换和支持的
You should write the pickled data with a lower protocol number in Python 3. Python 3 introduced a new protocol with the number 3
(and uses it as default), so switch back to a value of 2
which can be read by Python 2.
Check the protocol
parameter in pickle.dump
. Your resulting code will look like this.
pickle.dump(your_object, your_file, protocol=2)
There is no protocol
parameter in pickle.load
because pickle
can determine the protocol from the file.
Pickle uses different protocols
to convert your data to a binary stream.
You must specify in python 3 a protocol lower than 3
in order to be able to load the data in python 2. You can specify the protocol
parameter when invoking pickle.dump
.