It seems that you're using an outdated browser. Some things may not work as they should (or don't work at all).
We suggest you upgrade newer and better browser like: Chrome, Firefox, Internet Explorer or Opera

×
avatar
Kalanyr: I'm currently looking into what's going on with the mysterious failures to download files in largeish downloads. I think I've found out why it's so fragile, so I should be able to significantly reduce the instance of it happening. But I'll have to wait until I have a big download set to test it again, I used the last big lot with a heap of debugging outputs to work out where things were going wrong.
You could just move files and redownload if you don't have a data cap you're working against.
I've uploaded a new build that should hopefully not have downloads mysteriously terminate without trace, and should reduce (but probably not eliminate the number of AWOL threads, I can see why that's happening but I'm either going to have to restructure the code a fair bit or do a lot of very repetitive try/catch statements to fix that).

It also has more detailed logging of Exceptions in the Download Inner IO Loop and the Download worker thread, so if you have threads go missing / download items that don't complete without a sensible reason, please attach your logs to your error reports.
For those getting an error message about a missing module
pip install six cryptography
pip install pyopenssl

will do the trick.

@Kalanyr
Thanks for the updated file. I am currently running an update. Might also run a download this evening.
Ah.. I still used the previous version, but I didn't notice anything strange O_o
Checking the manifest took 40 mins for ~700 games.
Post edited January 23, 2018 by phaolo
avatar
phaolo: Ah.. I still used the previous version, but I didn't notice anything strange O_o
Checking the manifest took 40 mins for ~700 games.
Could you check the new version and see what your speed is like then please ? I just went back and checked an old version I know was really fast and and it was 5 hours for ~1300 games which is about the normal speed now, suggesting the extra wait is on GOG's side.
avatar
phaolo: Ah.. I still used the previous version, but I didn't notice anything strange O_o
Checking the manifest took 40 mins for ~700 games.
avatar
Kalanyr: Could you check the new version and see what your speed is like then please ? I just went back and checked an old version I know was really fast and and it was 5 hours for ~1300 games which is about the normal speed now, suggesting the extra wait is on GOG's side.
5 hours? O_o
Are you talking only about the command "update" too or are you considering the full process with all games missing?

I use this command for the manifest:
gogrepo.py update -resumemode resume -skiphidden -os windows -lang en

I can't test the new version for now, because I've already updated everything.
Post edited January 26, 2018 by phaolo
avatar
phaolo: 5 hours? O_o
Are you talking only about the command "update" too or are you considering the full process with all games missing?
What's the difference?

avatar
phaolo: I can't test the new version for now, because I've already updated everything.
That shouldn't really prevent you from testing, or? I presume you could try e.g. this:

1. Rename your current manifest file to something else (or move it elsewhere).

2. gogrepo.py update -lang en -os windows (so it creates a new manifest file from scratch)

3. How long does it take for you?

Afterwards you can copy over your old manifest file, if that is what you wanted to keep (e.g. due to the verification status). Anyway, I have also noticed that gogrepo update seems to have slowed down. For me it took like 3½ hours for 1418 games or so (Windows English); hopefully it hasn't slowed down even more, since Kalanyr says it took five hours for 1300 games... Anyway, even that is better what it used to be for me with the vanilla version of gogrepo, something like 10 hours or so.
Post edited January 26, 2018 by timppu
avatar
phaolo: 5 hours? O_o
Are you talking only about the command "update" too or are you considering the full process with all games missing?
avatar
timppu: What's the difference?

avatar
phaolo: I can't test the new version for now, because I've already updated everything.
avatar
timppu: That shouldn't really prevent you from testing, or? I presume you could try e.g. this:

1. Rename your current manifest file to something else (or move it elsewhere).

2. gogrepo.py update -lang en -os windows (so it creates a new manifest file from scratch)

3. How long does it take for you?

Afterwards you can copy over your old manifest file, if that is what you wanted to keep (e.g. due to the verification status). Anyway, I have also noticed that gogrepo update seems to have slowed down. For me it took like 3½ hours for 1418 games or so (Windows English); hopefully it hasn't slowed down even more, since Kalanyr says it took five hours for 1300 games... Anyway, even that is better what it used to be for me with the vanilla version of gogrepo, something like 10 hours or so.
It's very dependent on the games (the number of items and if they have verification data) and also the network (a lot of the wait is the ping since there's a lot of connections). My speed is unchanged since my last test in the same circumstance.
avatar
Kalanyr: It's very dependent on the games (the number of items and if they have verification data) and also the network (a lot of the wait is the ping since there's a lot of connections). My speed is unchanged since my last test in the same circumstance.
Ok. I'll probably test it anyway over the weekend, whether there has been any change.

Oh well, at least it works. If GOG feels they need to throttle the incoming http requests or something, so be it. I guess it is understandable.
avatar
phaolo: 5 hours? O_o
Are you talking only about the command "update" too or are you considering the full process with all games missing?
avatar
timppu: What's the difference?
The "full process" means also downloading the games, whilst "update only" means only modifying the manifest file.

avatar
phaolo: I can't test the new version for now, because I've already updated everything.
avatar
timppu: That shouldn't really prevent you from testing, or? I presume you could try e.g. this:
1. Rename your current manifest file to something else (or move it elsewhere).
2. gogrepo.py update -lang en -os windows (so it creates a new manifest file from scratch)
3. How long does it take for you?
Ah ok, I could try that.
(maybe tomorrow)
It would be nice for Jhonny or someone to confirm that GOG is or is not actively combating this so we can know that it will take forever rather than spending energy working around it. It would indeed save trouble on everyone's part.
avatar
timppu:
avatar
Kalanyr:
Uh.. guys?
I've tested the new version as instructed and.. for 711 games it took only 59 minutes.
Mostly the same speed of the previous one, without any big delay.
O_o
Post edited January 26, 2018 by phaolo
avatar
phaolo: Uh.. guys?
I've tested the new version as instructed and.. for 711 games it took only 59 minutes.
Mostly the same speed of the previous one, without any big delay.
O_o
I'll test it later, but that is not entirely in a different ballpark what I was getting before. I have about twice as many games, and for me it took 3½ hours. Ok, it is considerably slower than what you are getting (2 hours vs 3½ hours), but who knows, maybe your ping times are indeed better, or you happen to have the kind of games which get the info faster. I've noticed that myself too, some games get "updated" much faster than others, maybe related to how many downloadable items there are and shit.

Ps. What was the exact command you used, e.g. did you get extras too etc.?
avatar
timppu: What was the exact command you used, e.g. did you get extras too etc.?
I used the command that you told me:
gogrepo.py update -lang en -os windows
Odd, now it is again faster for me. It took me 1 h 9 minutes to get update for 1420 games. Also, unlike earlier when it was fast, I didn't get any "ban period" either, not sure if I was just lucky and I just went under the limit for the first hour or something...

Anyway, now it seems good to me, and hopefully it also stays this way. :)