suzanne.soy/racket
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Better randomization of poll dates -- multiply the range by a factor of 1..2.

Browse Source
This commit is contained in:
Eli Barzilay 2011-08-12 19:10:57 -04:00
parent 456e9befd3
commit 046817a328
1 changed files with 9 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
collects/meta/web/download/mirror-link.rkt
View File

@ -17,10 +17,10 @@ Polling a URL can result in one of four options:
minutes (or more) later. minutes (or more) later.
2. The URL exists, but no size information is available (via a HEAD 2. The URL exists, but no size information is available (via a HEAD
query, or via an FTP directory listing). The link will be shown in query, or via an FTP directory listing). The link will be shown in
this case, but it will be re-polled two days later. (With a random this case, but it will be re-polled two days later. (With a 1..2
factor, and a nightly build that happens at the same time, this might random factor, and a nightly build that happens at the same time,
mean more days.) So far, all mirrors provide size information, so this will be around 3-5 days.) So far, all mirrors provide size
this works fine. information, so this works fine.
3. The URL exists and we get its size, but the size does not match. The 3. The URL exists and we get its size, but the size does not match. The
URL is not shown, and will be re-polled in an hour. The assumption URL is not shown, and will be re-polled in an hour. The assumption
here is either bad synchronization, or we caught it in progress. here is either bad synchronization, or we caught it in progress.
@ -54,23 +54,22 @@ Polling a URL can result in one of four options:
(define entry (assoc url known-mirrors)) (define entry (assoc url known-mirrors))
(define last-time (and entry (cadr entry))) (define last-time (and entry (cadr entry)))
(define result (and entry (caddr entry))) (define result (and entry (caddr entry)))
(define R (+ 1 (random))) ; random 1..2 number, to avoid congestion
(define new (define new
(and (cond (and (cond
;; failed, check again after 15 minutes (to accomodate re-runs after ;; failed, check again after 15 minutes (to accomodate re-runs after
;; a release was done) ;; a release was done)
[(eq? #f result) [(eq? #f result)
(or (not entry) ; actually missing => try now (or (not entry) ; actually missing => try now
(current-time . > . (+ last-time (* 15 60))))] (current-time . > . (+ last-time (* 15 60 R))))]
;; known but without a size to verify, check again after two days ;; known but without a size to verify, check again after two days
[(eq? #t result) [(eq? #t result)
(and (current-time . > . (+ last-time (* 2 24 60 60))) (current-time . > . (+ last-time (* 2 24 60 60 R)))]
(zero? (random 3)))]
;; has a bad size, check again after an hour ;; has a bad size, check again after an hour
[(not (= result size)) [(not (= result size))
(current-time . > . (+ last-time (* 60 60)))] (current-time . > . (+ last-time (* 60 60 R)))]
;; otherwise check again after a month ;; otherwise check again after a month
[else (and (current-time . > . (+ last-time (* 30 24 60 60))) [else (current-time . > . (+ last-time (* 30 24 60 60 R)))])
(zero? (random 20)))])
(list url current-time (thunk)))) (list url current-time (thunk))))
(when new (when new
;; keep them sorted by time ;; keep them sorted by time