Tools to Capture and Convert the Web
GrabzIt's Online Community

Throw error when I run demo by python

error:

Traceback (most recent call last):

  File "GrabzItClient.py", line 6, in <module>

    grabzIt.SaveTo("documents/result.docx")

  File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 235, in SaveTo

    id = self.Save()

  File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 209, in Save

    obj = self._take(sig, callBackURL)

  File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 223, in _take

    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))

  File "C:\Program Files\Python36\lib\site-packages\GrabzIt\GrabzItClient.py", line 629, in HTTPPost

    h.endheaders()

  File "C:\Program Files\Python36\lib\http\client.py", line 1234, in endheaders

    self._send_output(message_body, encode_chunked=encode_chunked)

  File "C:\Program Files\Python36\lib\http\client.py", line 1026, in _send_output

    self.send(msg)

  File "C:\Program Files\Python36\lib\http\client.py", line 964, in send

    self.connect()

  File "C:\Program Files\Python36\lib\http\client.py", line 936, in connect

    (self.host,self.port), self.timeout, self.source_address)

  File "C:\Program Files\Python36\lib\socket.py", line 704, in create_connection

    for res in getaddrinfo(host, port, 0, SOCK_STREAM):

  File "C:\Program Files\Python36\lib\socket.py", line 743, in getaddrinfo

    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):

socket.gaierror: [Errno 11001] getaddrinfo failed

 

source：(I installed GrabzIt and got APPLICATION KEY and APPLICATION SECRET)

from GrabzIt import GrabzItClient

 

grabzIt = GrabzItClient.GrabzItClient(APPLICATION KEY, APPLICATION SECRET)

grabzIt.HTMLToDOCX("<html><body><h1>Hello World!</h1></body></html>") 

grabzIt.SaveTo("documents/result.docx") 

 

could you tell me what should I do ?

 Asked by anonymous on Tuesday, 21 August, 2018 06:54:17 AM

Welcome to the community!

Usually a socket.gaierror: [Errno 11001] getaddrinfo failed is caused when the code is being called from behind a firewall or proxy. Please ensure the application has access to port 80.

 Answered by GrabzIt Support on Tuesday, 21 August, 2018 07:01:51 AM

Thank you for the answer.

Yes,It's the problem of proxy.It runs OK when I change the other PC which have not proxy.

How can it work with proxy?

 Answered by anonymous on Tuesday, 21 August, 2018 09:30:49 AM

This will be a new feature. We are looking if we can alter the Python client to enable using proxy servers.

If it is possible we will get back to you with a fix. You can have a go yourself if you like as all of our clients are open source: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

 Answered by GrabzIt Support on Tuesday, 21 August, 2018 12:20:04 PM

We have added the new feature. Just download the GrabzItClient.py file and replace your copy from here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

You can then set the proxy url of your proxy server lusing the new SetLocalProxy method, for example:

grabzIt.SetLocalProxy("http://123.123.123.123:21231")

Please let us now how it goes and we will do a new release soon!

 Answered by GrabzIt Support on Tuesday, 21 August, 2018 05:38:30 PM

Thank you for the answer.

I replaced the GrabzItClient.py and changed my source :

 

 

grabzIt = GrabzItClient.GrabzItClient(APPLICATION KEY, APPLICATION SECRET)

grabzIt.SetLocalProxy("http://xxx@jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080")

grabzIt.HTMLToDOCX("<html><body><h1>Hello World!</h1></body></html>") 

grabzIt.SaveTo("documents/result.docx") 

 

user:xxx@jp.xxxxx.com

password:yyyyyyy

proxy server:zzz.proxy.zzz.zzzzzzz.com

port:8080

 

error message:

Traceback (most recent call last):
  File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 487, in SaveTo
    id = self.Save()
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 435, in Save
    obj = self._take(sig, callBackURL)
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 463, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 1307, in HTTPPost
    headers['Proxy-Authorization'] = 'Basic ' + base64.b64encode(auth)
  File "C:\Program Files\Python36\lib\base64.py", line 58, in b64encode
    encoded = binascii.b2a_base64(s, newline=False)
TypeError: a bytes-like object is required, not 'str'

 

same error message even I changed SetLocalProxy from "http://xxx@jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080" to "http://xxx%40jp.xxxxx.com:yyyyyyy@zzz.proxy.zzz.zzzzzzz.com:8080"

 

 Answered by anonymous on Wednesday, 22 August, 2018 12:34:43 AM

sorry,may be I set the wrong proxy.

I changed the proxy to "http://the_other_proxy_server:port" and error message became:

Traceback (most recent call last):
  File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 487, in SaveTo
    id = self.Save()
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 435, in Save
    obj = self._take(sig, callBackURL)
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 463, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 1327, in HTTPPost
    h.endheaders()
  File "C:\Program Files\Python36\lib\http\client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\Python36\lib\http\client.py", line 1026, in _send_output
    self.send(msg)
  File "C:\Program Files\Python36\lib\http\client.py", line 964, in send
    self.connect()
  File "C:\Program Files\Python36\lib\http\client.py", line 940, in connect
    self._tunnel()
  File "C:\Program Files\Python36\lib\http\client.py", line 919, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 407 Proxy Authentication Required

 Answered by anonymous on Wednesday, 22 August, 2018 12:40:52 AM

You need to specify your username and password like so: http://username:password@example.com:12335

 Answered by GrabzIt Support on Wednesday, 22 August, 2018 05:38:10 AM

Yes,I specified my username and password (question on Wednesday, 22 August, 2018 09:34:43 AM) like :

http://username:password@example.com:12335

username is Email address (xxx@jp.xxxxx.com) so I changed to xxx%40jp.xxxxx.com

but it's the same error message:TypeError: a bytes-like object is required, not 'str'

 Answered by anonymous on Wednesday, 22 August, 2018 05:50:49 AM

Sorry, I missed your previous response. I think this is a Python 3 issue and have released a fix that should solve the error. Please can you update GrabzItClient.py again from: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

 Answered by GrabzIt Support on Wednesday, 22 August, 2018 06:40:47 AM

I replaced python file and find new error message:

Traceback (most recent call last):

  File "GrabzIt_test.py", line 54, in <module>

    grabzIt.SaveTo("result.docx")

  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 244, in SaveTo

    id = self.Save()

  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 218, in Save

    obj = self._take(sig, callBackURL)

  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 232, in _take

    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))

  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 661, in HTTPPost

    headers['Proxy-Authorization'] = 'Basic ' + encodedAuth

TypeError: must be str, not bytes

 Answered by anonymous on Wednesday, 22 August, 2018 08:39:24 AM

Ok, we have decoded it into ascii. We aren't using Python 3, which is why it caught us out. Please update your code from here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

As this is open source, feel free to debug and commit any improvements you want to Github. As we don't have access to your setup.

 Answered by GrabzIt Support on Wednesday, 22 August, 2018 09:22:01 AM

Many thanks for your help.

OSError: Tunnel connection failed: 407 Proxy Authentication Required

I will try to debug. 

 Answered by anonymous on Wednesday, 22 August, 2018 10:23:31 AM

We think we know what the issue is, however it will require re-writing some of the code. So we will contact you when done. Hopefuly in 24 hours.

 Answered by GrabzIt Support on Wednesday, 22 August, 2018 12:15:22 PM

I believe we have fixed the issue! You can get the latest versiom here: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

 Answered by GrabzIt Support on Wednesday, 22 August, 2018 05:30:43 PM

Thanks for reply.

New error message:

Traceback (most recent call last):
  File "GrabzIt_test.py", line 54, in <module>
    grabzIt.SaveTo("result.docx")
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 243, in SaveTo
    id = self.Save()
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 217, in Save
    obj = self._take(sig, callBackURL)
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 231, in _take
    return self.HTTPPost(self.request.url, self.request.options._getParameters(self.applicationKey, sig, callBackURL, 'html', quote(self.request.data)))
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 629, in HTTPPost
    h = self._getConnection()
  File "C:\TEST\GrabzIt_python\GrabzIt\GrabzItClient.py", line 679, in _getConnection
    h.putheader('proxy-authorization', 'Basic ' + encodedAuth)
  File "C:\Program Files\Python36\lib\http\client.py", line 1201, in putheader
    raise CannotSendHeader()
http.client.CannotSendHeader

 

 Answered by anonymous on Wednesday, 22 August, 2018 11:52:17 PM

We have made some minor improvements to: https://github.com/GrabzIt/grabzit/blob/master/python/GrabzIt/GrabzItClient.py

And have tested it in Python 2.7, 3.4 and 3.6 and it works in all enviroments, with and without proxies and with authorized proxies.

If it is still not working for you, you will need to debug the code.

 Answered by GrabzIt Support on Thursday, 23 August, 2018 11:41:44 AM

Thank you for reply.

I will debug my source.

By the way, did you tested the username which is Email like xxx@yyy.com ?

 Answered by anonymous on Thursday, 23 August, 2018 11:56:46 AM

No, I didn't that would need to be properly encoded and decoded so maybe that is the problem.

 Answered by GrabzIt Support on Thursday, 23 August, 2018 11:59:36 AM
Answer QuestionView all Web Capture Questions
Try all our premium features for free with a 7 day free trial. Then from $5.99 a month, unless cancelled.
  • More Captures
  • More Features
  • More API's
  • Bigger Scrapes
  • Bigger Captures
  • Bigger Everything
Start Free Trial


Terms and Conditions
Privacy Policy
© GrabzIt® 2019