Throw error when I run demo by python

error:

Traceback (most recent call last):

File "GrabzItClient.py", line 6, in <module>

grabzIt.SaveTo("documents/result.docx")

File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 235, in SaveTo

id = self.Save()

File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 209, in Save

obj = self._take(sig, callBackURL)

File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 223, in _take

return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))

File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 629, in HTTPPost

h.endheaders()

File "C:\Program Files\Python36\lib\http\client.py", line 1234, in endheaders

self._send_output(message_body, encode_chunked=encode_chunked)

File "C:\Program Files\Python36\lib\http\client.py", line 1026, in _send_output

self.send(msg)

File "C:\Program Files\Python36\lib\http\client.py", line 964, in send

self.connect()

File "C:\Program Files\Python36\lib\http\client.py", line 936, in connect

(self.host,self.port), self.timeout, self.source_address)

File "C:\Program Files\Python36\lib\socket.py", line 704, in create_connection

for res in getaddrinfo(host, port, 0, SOCK_STREAM):

File "C:\Program Files\Python36\lib\socket.py", line 743, in getaddrinfo

for res in _socket.getaddrinfo(host, port, family, type, proto, flags):

socket.gaierror: [Errno 11001] getaddrinfo failed

source：(I installed GrabzIt and got APPLICATION KEY and APPLICATION SECRET)

from GrabzIt import GrabzItClient

grabzIt = GrabzItClient.GrabzItClient(APPLICATION KEY, APPLICATION SECRET)

grabzIt.HTMLToDOCX("<html><body><h1>Hello World!</h1></body></html>")

grabzIt.SaveTo("documents/result.docx")

could you tell me what should I do ?

Asked by anonymous on the 21st of August 2018

Welcome to the community!

Usually a socket.gaierror: [Errno 11001] getaddrinfo failed is caused when the code is being called from behind a firewall or proxy. Please ensure the application has access to port 80.

Answered by GrabzIt Support on the 21st of August 2018

Thank you for the answer.

Yes,It's the problem of proxy.It runs OK when I change the other PC which have not proxy.

How can it work with proxy?

Answered by anonymous on the 21st of August 2018

This will be a new feature. We are looking if we can alter the Python client to enable using proxy servers.

If it is possible we will get back to you with a fix. You can have a go yourself if you like as all of our clients are open source: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

Answered by GrabzIt Support on the 21st of August 2018

We have added the new feature. Just download the GrabzItClient.py file and replace your copy from here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

You can then set the proxy url of your proxy server lusing the new SetLocalProxy method, for example:

grabzIt.SetLocalProxy("http://123.123.123.123:21231")

Please let us now how it goes and we will do a new release soon!

Answered by GrabzIt Support on the 21st of August 2018

Thank you for the answer.

I replaced the GrabzItClient.py and changed my source :

grabzIt = GrabzItClient.GrabzItClient(APPLICATION KEY, APPLICATION SECRET)

grabzIt.SetLocalProxy("http://xxx@jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080")

grabzIt.HTMLToDOCX("<html><body><h1>Hello World!</h1></body></html>")

grabzIt.SaveTo("documents/result.docx")

user:xxx@jp.xxxxx.com

password:yyyyyyy

proxy server:zzz.proxy.zzz.zzzzzzz.com

port:8080

error message:

Traceback (most recent call last):
File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 487, in SaveTo
    id = self.Save()
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 435, in Save
    obj = self._take(sig, callBackURL)
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 463, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 1307, in HTTPPost
    headers['Proxy-Authorization'] = 'Basic ' + base64.b64encode(auth)
File "C:\Program Files\Python36\lib\base64.py", line 58, in b64encode
    encoded = binascii.b2a_base64(s, newline=False)
TypeError: a bytes-like object is required, not 'str'

same error message even I changed SetLocalProxy from "http://xxx@jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080" to "http://xxx%40jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080"

Answered by anonymous on the 22nd of August 2018

sorry,may be I set the wrong proxy.

I changed the proxy to "http://the_other_proxy_server:port" and error message became:

Traceback (most recent call last):
File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 487, in SaveTo
    id = self.Save()
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 435, in Save
    obj = self._take(sig, callBackURL)
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 463, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 1327, in HTTPPost
    h.endheaders()
File "C:\Program Files\Python36\lib\http\client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Program Files\Python36\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
File "C:\Program Files\Python36\lib\http\client.py", line 964, in send
    self.connect()
File "C:\Program Files\Python36\lib\http\client.py", line 940, in connect
    self._tunnel()
File "C:\Program Files\Python36\lib\http\client.py", line 919, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 407 Proxy Authentication Required

Answered by anonymous on the 22nd of August 2018

You need to specify your username and password like so: http://username:password@example.com:12335

Answered by GrabzIt Support on the 22nd of August 2018

Yes,I specified my username and password (question on Wednesday, 22 August, 2018 09:34:43 AM) like :

http://username:password@example.com:12335

username is Email address (xxx@jp.xxxxx.com) so I changed to xxx%40jp.xxxxx.com

but it's the same error message:TypeError: a bytes-like object is required, not 'str'

Answered by anonymous on the 22nd of August 2018

Sorry, I missed your previous response. I think this is a Python 3 issue and have released a fix that should solve the error. Please can you update GrabzItClient.py again from: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

Answered by GrabzIt Support on the 22nd of August 2018

I replaced python file and find new error message:

Traceback (most recent call last):

File "GrabzIt_test.py", line 54, in <module>

grabzIt.SaveTo("result.docx")

File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 244, in SaveTo

id = self.Save()

File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 218, in Save

obj = self._take(sig, callBackURL)

File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 232, in _take

return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))

File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 661, in HTTPPost

headers['Proxy-Authorization'] = 'Basic ' + encodedAuth

TypeError: must be str, not bytes

Answered by anonymous on the 22nd of August 2018

Ok, we have decoded it into ascii. We aren't using Python 3, which is why it caught us out. Please update your code from here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

As this is open source, feel free to debug and commit any improvements you want to Github. As we don't have access to your setup.

Answered by GrabzIt Support on the 22nd of August 2018

Many thanks for your help.

OSError: Tunnel connection failed: 407 Proxy Authentication Required

I will try to debug.

Answered by anonymous on the 22nd of August 2018

We think we know what the issue is, however it will require re-writing some of the code. So we will contact you when done. Hopefuly in 24 hours.

Answered by GrabzIt Support on the 22nd of August 2018

I believe we have fixed the issue! You can get the latest versiom here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

Answered by GrabzIt Support on the 22nd of August 2018

Thanks for reply.

New error message:

Traceback (most recent call last):
File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 243, in SaveTo
    id = self.Save()
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 217, in Save
    obj = self._take(sig, callBackURL)
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 231, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 629, in HTTPPost
    h = self._getConnection()
File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 679, in _getConnection
    h.putheader('proxy-authorization', 'Basic ' + encodedAuth)
File "C:\Program Files\Python36\lib\http\client.py", line 1201, in putheader
    raise CannotSendHeader()
http.client.CannotSendHeader

Answered by anonymous on the 22nd of August 2018

We have made some minor improvements to: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

And have tested it in Python 2.7, 3.4 and 3.6 and it works in all enviroments, with and without proxies and with authorized proxies.

If it is still not working for you, you will need to debug the code.

Answered by GrabzIt Support on the 23rd of August 2018

Thank you for reply.

I will debug my source.

By the way, did you tested the username which is Email like xxx@yyy.com ?

Answered by anonymous on the 23rd of August 2018

No, I didn't that would need to be properly encoded and decoded so maybe that is the problem.

Answered by GrabzIt Support on the 23rd of August 2018

Answer Question View all Web Capture Questions