This sounds like a better option than the other poll.. are the two solutions mutually exclusive?

^I'd agree as well. Worse case is that some people's videos get voted on again, but that shouldn't matter that much in the long haul, as the numbers will be closer to the original vote count than they are now.

I did not see this poll before voting on the other one. This is much better.

Oops, I accidentally voted no when I meant yes. Sorry.

OK I am confused. (stupid)
You possess data that will let you restore all vote counts to how they were prior 3/11? And that is not relying on any google cache data?
I'm just talking about vote numbers .. not users who cast them.
If that's the case then this is a no brainer.
We are all going crazy trying to re-create votes! I don't think a single one of us gives a shit about being recognized for voting at this point. This shouldn't even be up to a vote imo. Those votes that were cast are a part of the backbone of this site and though I say they don't matter in trying to help us move forward .. they most definitely did entirely shape our past and should be restored without a doubt.

If we go with this, will there be a problem with all the votes cast since 3/11? I've been upvoting old vids like a mofo, and I don't want my votes counted twice, and I'm sure no one else does either. That's my one and only concern.

I'm willing to lose the last day or so data in voting etc to regain all the rest.

No need to lose any new votes. As mentioned above, that's the one drawback of this fix: Unknown voters would be turned into siftbot (most likely) so that means the original unknown voters could vote again.

Awesome. I'm asking though .. you can restore all votes prior to 3/11 without relying on google cache etc? You have that data?
The other poll made it a little confusing .. referencing that the data was old and/or cobbled together from google cache etc.

But this one states:
"We still possess all vote *counts* for every video up to the crash of 311, meaning the actual number of votes received for each video."

and if that's the case then what the hell are we waiting for.

For me to implement then run it I guess.

Speaking of that, the run will take a very long time and seriously tax the site so we may head back into Razzleberry chat while we do it.

And to clarify, yes, we have all vote totals forever, but not all voter info. We haven't done anything with that info before now because it just hit us that it's available.

I bloody voted no when I meant yes too! I say crack on fellas

If you have all the vote totals and that data is from your shit and not google or yahoo then you should restore it immediately. Fuck voting on it. This is about restoring the site and all of it's votes.

How are you ever going to reconcile not giving all of those votes back .. when you have that data?

Aparently I'm the only drunk here who hasn't voted yet

>> ^kronosposeidon:
Oops, I accidentally voted no when I meant yes. Sorry.


>> ^alien_concept:
I bloody voted no when I meant yes too! I say crack on fellas

We're not not giving those votes back. Calm down, tiger.

This is all breaking news coming out as it's happening. I just posted the poll as the realization hit me and am leaving it here so everyone knows what's going on.

I have no idea what's going on .. what votes aren't you giving back? What I meant by that was, if the poll goes negative .. and you don't restore those votes .. when you have that vote data .. how are you going to say that's OK?
You're basically saying even though we know all the vote totals and how they should be .. we won't restore them based upon some poll.

I only fear that this creates even more inflation in the votes, if many people stumble back and vote again. But it's much better to have that than to lose all the rankings. The video rankings are more important.. do it.


But, for the love of god, make it very clear which votes are from before 3/11, so it's clear to new voters what has happened.

Also, could I request that they not be given back to the ukulele video?

When did I say that?

Once again:
I posted this poll in the heat of the moment right when the idea hit me. I made it a poll because I was in poll mode after just finishing submission of the other poll (that I discarded). I am leaving this poll here so everyone knows what's going on.

Okay, you're turn again. I'll keep responding instead of fixing the votes until I can successfully convince you that I've answered your question.

I think the poll should be re-structured.

a. restore all votes
b. continue catastrophe

dude I don't understand but I will just vote yes restore votes and shut the fuck up .. because if me babbling is any way stopping you from doing your thing and getting shit done I don't want to do that ..

I think your subconscious mind doesn't want me to fix the votes otherwise you wouldn't keep me preoccupied reading your responses.

Whatever .. you know I love the sift.
I ain't too stressed over votes personally and I've tried to de-stress others about it.
but I don't get what you are saying right now about "fixing" votes and how I'm stopping you.
Are you doing something regardless of this poll to restore votes that somehow my replies are interfering with?

I'd like to cast my vote for a giant squid teleported into the middle of the internet to unite us all into one, peaceful website.

One website where we can all get along, vote on shit, and watch porn. Divided we fall, united we fap.

Burd, try to read his posts again.

What's happening is:

the votes cast after 3/11 stays

a) the votes before 3/11 will be restored, but only the total counts. Individual voter names will not be restored, dummies will be used.

b) vote counts are not restored. It stays like it is now.

c) lucky/dag manually restores some votes, complete with names.


C is obviously a poor solution, because of the obvious unfairness factor ("what about MY video??").
A will restore the main function of the sift, by sorting / sifting the videos again, but creates a double-vote effect where someone has voted twice (before /after 3/11).
B will lose the original sortings, but not dupe-votes.

I'm guessing you are already getting this, and you don't follow why A isn't just done, is that correct? If it is, then there are downsides to A too, so I, for one, think it is a good idea to ask the community, which solution is the one we should use.

A is by far the best, in my opinion though.

This is very dangerous. I know that everyone wants their vote tallies back, but combining the old vote counts with all of the new voting on old videos that has occurred will skew things even more than what you suggested in the other poll (restoring all vote data up to 10/2/2007).

I'm only using the following as an example, so don't flame me for it. The entirety of IssyKitty's sifted videos were voted up by dft and gratefulmom after the crash. If you restore the old vote counts with those votes added, you will have effectively given her an extra 3000 votes or so. That hardly seems right, especially considering that there are many other users whose videos have barely been noticed since the crash.

The point is, if you restore the old vote counts while still allowing votes from anyone on those old videos, it will only reinforce cliquish behavior - "I'll upvote all your old vids if you do mine." I know that sounds petty, and I'd like to think that everyone here is above that, but clearly we are not if the recent voting patterns are any indication.


I think restoring the complete vote data up to 10/2007 is good idea on its own. I can see how a lot of more recent members might object to this, but if you have complete and correct data up to that point, it seems a no-brainer to restore what you've got. Restoring partial data on everything, though, will ultimately lead to greater bias on videos that don't necessarily deserve it. And the reverse is true as well - videos that were already obscure will get their 9 votes back and never see the light of day again unless some angel promotes them.


Is there anyway to freeze voting on pre-crash vids if you do this and throw out votes on those vids post crash? Obviously that has the flaw of not allowing any new original votes being cast on older vids, but without such a quarantine, I don't see how this benefits the functioning of the site. The whole point is to reward quality videos and make the best of them easier to find. If the old vote counts are reinstated with no restriction on new votes, you will be adding to any submitter-based bias for every video and a date bias for any video submitted prior to 3/11.


[edit] Apparently I inadvertently clicked on one of the voting options in this poll without realizing it. I was going to abstain, but if I have to vote, count me for option 3, restore all completely correct data up to 10/2/2007. It is the only option which will allow the other potential remedies to be used in the future.

>> ^jonny:
This is very dangerous.

Oh fuck! Oh fuck! Run, run, RUN!! I didn't sign up for this!!

>> ^gwiz665:
C is obviously a poor solution, because of the obvious unfairness factor ("what about MY video??").

But this is the only non-destructive option which restores any data and allows for potential future fixes. It's the only one that makes sense to me (as far as the given options go).


A will restore the main function of the sift, by sorting / sifting the videos again, but creates a double-vote effect where someone has voted twice (before /after 3/11).

This is by far the most potentially destructive, because it specifically allows voting twice on vids. This goes both ways too - downvotes as well as upvotes. Do you really want people having the option of downvoting your barely sifted vids again? Do you want top 15 vids getting even that much more attention when their vote counts are already skewed?


B will lose the original sortings, but not dupe-votes.

I think it loses more than just the original sorting, but it is probably the "safest" path.

why can't i vote on this?

>> ^Edeot:
>> ^jonny:
This is very dangerous.
Oh fuck! Oh fuck! Run, run, RUN!! I didn't sign up for this!!


Well, let's see - you joined today, so you're either a sock puppet, a lurker, or completely irrelevant. Best guess is option number 1, but in any case, unless you actually have something useful to add, let me offer you this tall glass of STFU.

wtf are we talking about? That a few days of feverish voting by people who were trying to restore this site and their own years of posts is a problem? Is that the danger? Some double votes that I doubt would even happen? Who gives a SHIT! Restore the several years of vote data please. Having to worry about losing every vote ever cast by anyone ever since the sift became self aware vs oh shit some people may make out real good over the last couple of days ..
seriously ..

"The entirety of IssyKitty's sifted videos were voted up by dft... ...after the crash."


And before.

There's only one video I care about in my collection and either way would deal with it about as well so I'll abstain and let those with more of a stake in it make the choice.

I do want to say thanks to Lucky and anyone else working on this, for reasons I've whined enough about elsewhere that one video was kind of important to me. Thanks

Is it possible to make the old vote totals a ceiling which wouldn't grow any larger until new votes exceeded the old totals? Not sure if that would be hard to do, but it would be a happy medium and would prevent jonny from having nightmares about issy and I.

>> ^jonny:
>> ^Edeot:
>> ^jonny:
This is very dangerous.
Oh fuck! Oh fuck! Run, run, RUN!! I didn't sign up for this!!

Well, let's see - you joined today, so you're either a sock puppet, a lurker, or completely irrelevant. Best guess is option number 1, but in any case, unless you actually have something useful to add, let me offer you this tall glass of STFU.


I'll take an order of option 2 with a heaping helping of option 3. I'm gonna need a refill of STFU to wash it all down.

Relax, buddy. Twas a joke. Not trying to offend.

As usual fletch, you cut right to the chase.


No, burd, the danger isn't because of the activity in the last day and a half, but the potential for abuse in the future. The last day is just the beginning of skewing things if we go with door #1. That's exactly the point. Look how skewed things have gotten in just one day. Think about how biased vote counts on videos will be if we allow second votes on all videos submitted prior to 3/11. The whole point of voting on videos is to have the vote count in some way reflect the communal acknowledgement of quality. It's already biased enough as it is - why would we want to allow further bias to it?

[edit] More to the point, door #3 restores a very large amount of communal "knowledge" without adding new bias and without preventing any future fixes.

As ox pointed out, the votes ARE the information that is this site. the votes are "it".
All the vids arent even here, they are elsewhere. this is a repository of links. But links and votes.
Yes, any last few days voting will increase vote counts, but a true restoration to pre crash is best, and i'd vote for it, but i cant see the poll.


I wouldnt ask that the last few days revotes be removed from the resulting count increase, as they could be redone post this anyway and would be a pain to do.

A few vids with a few more votes than they should have is a tiny price to pay for all vids having atleast the count they should have.

I don't see how anything got skewed .. unless by skewed you mean started over. But what exactly got skewed by that? The original votes? OK .. well here they are apparently being offered to us. From what I understand they are saying they can restore the old votes. What's the problem? Why do we have to assume there will be second / double votes? I hate to keep proving my ignorance by asking these questions. Honestly in some ways it doesn't matter if I'm right or you are or someone else is. Those votes were cast by many people over many years and they are part of the history and blood of this site .. and they should stand and the site should be restored with them.

I am not ttaooly sure if I understoood this.. it is 5am, I have a dog that's had diarrhea the last day, so I am up all night trying to wipe squirts of poo of all surfaces..

If you can restore the vote count up until 3/11, please YES! I don't need names, just the count. As a newer member, I started using VS10 months ago. If data from earlier than that is brought back, most people will get their early votes, at least some thousands.. I'd lose the 23 000 votes I gained during my 10 months of activity and would be starting from scratch.

Just make sure the total vote count after the restore doesn't go way above the vote count on 3/10.

OK, I am being selfish again.. but like I said, I don't want to go to the bottom of the list just because I am a newer member and have been here for a short time

If I drop from being user #10 and having had lots of good vides that got 100+ votes, I think those deserve to be noticed. I don't want to drop to being user #300 with every video below 10 or 20 votes. I might as well give up then.
If you restore votes from years ago, just then give us 'new' users who were created later our vote counts. Or something.

Sorry. Sigh.

maybe it's just me, but i never thought the votes were the "backbone" of the site, and i tend not to care about old sifts.

Fuck it, everyone should just register on Facebook, Lucky should write an FB application, and we encase this old place in concrete. It was kinda shit before the crash and fixing it is just making it shitter. I thought the idea of the site was to replace TV, not to become a repository of clique backscratching activity, or a bandwidth sucking expensive monster.

Let's take a deep breath and move on, because in 5 years nobody will give a shit who upvoted what in 2006, and this is a chance for a fresh start.

After the Great Fire of London, Sir Christopher Wren made a beautiful new plan for rebuilding the city from scratch... but people couldn't agree about changing property rights, so they rebuilt on the old layout and made a fucking mess.

How about we just get a dose of heroin instead of our votes back?

>> ^burdturgler:
I don't see how anything got skewed .. unless by skewed you mean started over. But what exactly got skewed by that? The original votes? OK .. well here they are apparently being offered to us. From what I understand they are saying they can restore the old votes. What's the problem? Why do we have to assume there will be second / double votes?

Are you blind? Did you not notice the difference in the Top 15 sifters box? Or the vote counts on your own vids? If I understand Lucky correctly and option A is chosen, all of those new votes do not go away. They will be added to each video's vote count along with any restored vote counts. But more importantly, from here on out, any new votes on pre-crash videos will also be added. This will not be limited to a "handful" of videos. Personally, I've already voted on close to 1400 videos since the crash. And I'm one of the lazier voters. We're talking about quite literally 10s, maybe 100s, of thousands of votes being counted twice. And that's just the votes added in the last couple of days. Now imagine that compounded by all of the votes on those pre-crash videos for the life of VideoSift. Theoretically, every video sifted prior to 3/11 can at least double its vote count. But, of course, it won't happen evenly, or in some nice gaussian distribution. The pre-crash vids that get those second votes will belong to the most popular members, and not be cast based upon video quality, because that is how we will go about finding them. A couple of users (schmawy, KP) have admirably chosen more noble paths to redistribute their votes, but not only is that the exception, it is also unlikely to last.

Just to be clear - I'm am not making this argument to save my rank. If anything, my rank will go up if option A is chosen. I'm trying to preserve any semblance of what web 2.0 communal knowledge is about. If we restore all the previous vote counts without restoring any voter data, it's completely meaningless, and will only bias the votes even more than they already were.

Here's the reply I so meticulously crafted before the site was taken down (in my opinion a little too fast). Take it for what it's worth:


I am principally on jonny's side here, as the possible double-votes irk me to no end. Please don't take any hastened conclusions here before all the negative sides have been explored.

Burdturgler, you ask why we have to assume there will be double votes. Jonny has just provided an example with dft and Issykitty. I'm not wagging any finger here, but it shows how doublevoting might very well become an issue. In this case it has much more effect on user rankings than video rankings, and I believe that will be the biggest effect if this is implemented. Now if you like me believe that the videos rankings are more important you might be willing to pay that prize. But don't pretend that it's not an issue.

How doublevoting might affect the videos themselves is a murky issue, as I doubt that many people will have kronosposeidon's stamina to trek through all the >3000 pages of videos. The old videos that had hundreds of votes will likely not be affected much, as any doublevotes they might receive will be like drops in the ocean, and not all the say 280 voters will bother to dig it up and vote again. But for videos in the 10-50 range it might have a big effect. There is a huge difference between 20 and 40 in terms of perceived quality, so if a video is bumped that high it will give a skewed view of its quality as defined by the community.

A more immediate effect that I foresee if the vote counts are restored is this: Practically all the pqueued videos with 9 votes will get a visit from a friend that already voted for it pre-3/11, and get bumped into the sifted area. This will skew BOTH video rankings and user rankings. If you are ok with that, fine, but please take it into consideration.


The best solution as I see it is dystopianfuturetoday's suggestion: Let the old vote tallies be set as a threshold for the old videos that all new votes count towards, and which must be met before new votes start to pile on top. The effects I see from this is as follows:


- Videos with an awful lot of votes will practically never meet their threshold, and will retain their pre-3/11 vote counts indefinitely, unless their is a huge increase in membership.

- Videos with only a lot of votes will be similar, but might have a chance of meeting their threshold and becoming ordinary again if promoted.

- Videos with few votes and pqueued videos will have a big chance of meeting their threshold, so for all purposes they will be almost unaffected by the siftsplosion.

Videos that have not yet met their threshold should have this clearly stated on them, so people don't get confused as to why their voting doesn't work. Maybe they should even a small meter that shows its progress towards the threshold. I know I'm dumping a lot of coding work on lucky here now, but this is as I see it the most thorough way of salvaging our loss. I'm not able to vote in the poll for some reason, but count this as my vote.


Lastly, let me add that video rankings will always be biased towards newer videos, since there is substantial vote inflation as the membership increases. Also, I don't bother so much about the internal ranking of the most popular videos. A lot of votes IS a lot of votes, and it has proven its worth to the community just by getting so high. Bothering about the total vote tallies for these is therefore very unimportant, IMHO.

And a boon to you, burdturgler: With this solution you can promote The Dramatic Chipmunk again and again, until it surpasses The Infamous Ukulele video which will never reach its threshold!

Thanks for restoring the votes. But as a strange consequence I now have a video with 11 votes in my pqueue: http://www.videosift.com/video/Proofing-Fermat-s-Last-Theorem
Maybe there are others videos with where sifted due to siftbots voting spree

EDIT: Found two in the beggars canyon:
http://www.videosift.com/video/Al-Di-Meola-Egyptian-Danza-Live
http://www.videosift.com/video/THE-STRANGEST-DREAM

OMG! the votes are back atm! Still probably some minor adjustments need to be made, since for example I am now #8, above ant and Farhad, which was not true before..

But still, AMAZING job, I am realieved, happy and so tired I am either going to cry, or go back to sleep, or not.. hopefully there will be no more dog explosive farting.. she should have NOTHING in her tummy/bowels any more.. poor puppy

But thank you, thank you, thank you! Lucky and dag, I absolutely adore you. Thank you for all the work you put into fixing things..

OK, back to sleep now!

I ended up with a quirk or two also, but in all it's a great job and I'm glad to see this much. It's better than anyone expected just a few hours ago.

As far as I can figure the older stuff was restored intact but without newer votes, newer stuff put together best they could. That gives a couple of odd results it seems but for the one I cared about I never expected to see 10 votes on it again and it's got 53 of the 62 back. I'm happy enough given how in the dumps I was about it before.

I think the best way to fix the double vote problem is to just lock all pre 3/11 videos from new votes, and ad a message saying why. Old videos don't get that many extra votes anyway. From there on we can continue business as usual.

Comment hidden because you are ignoring dag. (show it anyway)

We'll still be doing some minor tweaks over the next couple of days - but the majority of the votes are back.

Great thanks to Lucky, Deathcow and Campiondelmundo who did some amazing data gymnastics to bring us back from the brink.

I raise my glass to you guys.

thanks lucky and dag, deathcow and campiondelmundo, you guys are great!

I would like to thank all who worked so hard to fix all of this..just wanted to mention that I have a few vids in my que as well that have 10 votes, just some info to help with the tweaking.

Cool, my avie is back too...:D

oh this looks pretty good who cares about double votes, if i was charter i would want my shit back. it's better than nothing and it probably took about 3 years off lucky's life so let's chill. but i still want a facebook app...

I'm just new to this. So vote counts have been restored and now doublevotes are likely to occur? I thought one of the core principles behind the sift was one vote per video.
I suppose this is the least worst scenario for those who really wanted the vote tallies back but still...

Great thanks to Lucky, Deathcow and Campiondelmundo who did some amazing data gymnastics to bring us back from the brink.
I raise my glass to you guys.


It's great that the votes have been restored, but I had no hand in that. Nice work lucky (and deathcow)! The sift looks alot like home again

YAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAY!!

Great work. Especially the part where you can't prove that I ever voted for a cat fart video.

This is awesome -- now I can DOUBLE UPVOTE all my most favourite vids!!!

Have downvotes been restored too? I know for a fact that some haven't been restored, because some of my videos that had downvotes now don't have them.

On a separate note, both Fedquip and mlx look like they've been shorted on votes. Before 3/11 they both had over 20,000 votes, but now neither do.

*promote, until more of these issues get ironed out.

Only published, queued, or discarded videos, published Sift Talk posts, and published blog posts may be promoted - ignoring promote request by kronosposeidon.

^Damn you to hell, tin can.

Awesome job recovering the votes .. thank you to everyone involved!
Definitely some details to iron out like KP said. I know that this had 2 downvotes that aren't there anymore. One of them was from a now banned user, not sure if that helps.
Regardless .. I think the Sift feels much more like home again.

I'm guessing the restored vote totals were an aggregate of both up and down votes, with no way of knowing specifically how many up or down votes were cast.

Regarding downvotes, keep in mind how our fix worked.

The only information we had is the number in the box next to a video. That number can be a result of x upvotes minus y downvotes = z net votes, but we don't have any up/down vote or voter info, just that number in the box. So all we did was add siftbot votes until z votes was achieved for the post.

>> ^lucky760:
all we did was add siftbot votes until z votes was achieved for the post.

Apparently you did more than that. It looks like you used the complete voter data for any posts submitted prior Sept 2, 2007, but didn't combine it with the more recent vote totals for those videos. The result is that any votes cast after 10/2 on videos submitted before that date are still missing. This video is a perfect example. It was submitted on 10/1, but didn't get published until several days later. So, it is now in my sifted list with only the 3 votes it got up to 10/2.

Any chance we can get the rest of those "siftbot" votes on videos submitted prior to 10/2/2007?

I realized that could be a problem but didn't realize to what extent. I didn't want to continue sticking my fingers into this and risk getting everything dirty, but I've gone ahead and done another update to hopefully fix that problem.

Hopefully all's straightened out now.

Sweet - thanks lucky. That definitely makes things better.

There's still some anomalies on some videos. For example, I have no idea how the votes on this video got the way they are. I had already voted for it prior to the crash, and I'm pretty sure I did not do so again afterwards. But somehow my vote is explicitly there. Overall, though, I think these are much fewer and further between now.