Recently I have been working with the Python requests module to secure an API call using the server’s certificate. I was stuck at a point and learning what I did to fix that issue was great and led me to create a post on . deep dive with SSL certificates The Problem I was using the requests module and here is the API call. response = requests.post(url, files=files, headers=headers) This is the error I was getting in return: / (Caused by SSLError(SSLCertVerification(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')) Attempts Doing unsecured calls with verify=false My first try was to use the verify flag as and try. False response = requests.post(url, files=files, headers=headers, verify=False) Though I got a 200, I got a nasty warning confirming that I am doing a horrible job, not providing a certificate. So I had to find the right way to do it. Provide the server certificate First I thought, if I can provide the server certificate in the verify key, it would do the trick. So I did, response = requests.post(url, files=files, headers=headers, verify='server.cer') This is a encoded certificate DER I got another error: / (Caused by SSLError(SSLError(136, '[X509] no certificate or crl found (_ssl.c:4232)')) Override CA_REQUESTS_BUNDLE The module requests to use to access the CA bundle and validate secure SSL connections and we can use the environment variable to override the CA bundle location. So I thought, if I can manually provide the in that variable, I will achieve enlightenment. But to my despair, that too failed. certifi CA_REQUESTS_BUNDLE server.cer Convert to different certificate encoding Then I thought that the module must not be taking the DER encoded certificate. So I converted to PEM which is plaintext but Base 64 encoded. requests openssl x509 -in server.cer -inform DER -outform PEM -out server.pem and did the call again response = requests.post(url, files=files, headers=headers, verify='server.pem') Another error. / (Caused by SSLError(SSLCertVerification(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')) Yes, this is the same as . So we are back to square 1. I tried to search the internet, but no one had a meaningful solution. Someone Error 1 Then I thought this is not the way, I will solve this issue. So I did a lot of studying for the Certificates and finally, I got them. I wrote the Deep Dive article and put everything there. Back to basics The problem and all the similar certificate-related issues (in any language not only python) on the internet require a clean and clear understanding of the certificate chain. I have very clearly explained everything in my deep dive post. Typically the certificate chain consists of 3 parties. A root certificate authority One or more intermediate certificate authority The server certificate is asking for the certificate to be signed. The delegation of responsibility is: Root CA signs → intermediate CA Intermediate CA signs → server certificate The Root certificates from Root CAs typically have a very long expiry date (more than 20 years) and come bundled as CA bundles in all the computers and servers and are kept very very securely under strict rules so that no one can alter them in any machine. As Root CA are very very sacred, they need intermediary CAs to delegate responsibility to sign a server certificate when anyone asks for it by providing a CSR. These intermediaries are called Intermediate CAs. There may be multiple intermediate CAs in a certificate chain. Solution In our case, when we converted the cert file to PEM format we do the error, unable to get local issuer certificate (_ssl.c:1108) This happens for 2 reasons: The intermediate CA certificate is not available on the file server.pem As we are manually specifying which certificate file to use by specifying *,* the python request module will not use the already existing CA bundle, rather will use the only and verify=server.pem server. pem expects that, it contains all the certificates in the chain, the server.cer, intermediate cert, and root cert So I manually stripped the server certificate like this: This is done in windows but, similar things can be done in Mac or Linux. After stripping all the certificates from the server.cer, we will have different .cer files for all the CAs. So for the above case, we will have 4 .cer files. Root CA(Zeescalar root ca) Intermediate CA 1(Zscalar intermediate Root CA) Intermediate CA 2 (Zscalar intermediate Root CA) google server .cer file Now, all we have to do is to convert all these files to files and add them together to create a consolidated file and feed it to python requests. .cer .pem pem So for all the cer files run the following command 4 times. openssl x509 -in server.cer -inform DER -outform PEM  >> consolidate.pem All we are doing it here is to create a full fledged CA bundle which has all the certificates and anyway we can do it, is just fine. That’s it, we feed our new CA pem file to python requests and it is happy. response = requests.post(url, files=files, headers=headers, verify='consolidate.pem')<Response [200]> Also published . here

Too Long; Didn't Read

Making security fun one episode at a time

Solving the Dreadful Certificate Issues in Python Requests Module

Too Long; Didn't Read

Supratim Samanta

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Solving the Dreadful Certificate Issues in Python Requests Module

Too Long; Didn't Read

Supratim Samanta

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES