Need Help/Advice re: robot.txt

[amydambly.livejournal.com]

Does anyone have any idea why a link would work one day on my computer but not the next day. I have had NO success what-so-ever in getting any of my saved links to fics at Wayback Machine. I don't know if there's a block or something on my computer. It keeps mentioning robot.txt.

I apologize if this isn't an appropriate place to post this but, since the links are all for B/J fics that aren't working I thought I'd at least try.

Thanks!!

Comments

[rinmonsterer.livejournal.com]

robot.txt is a file which tells robots - like google spider robot and archive robots like wayback machine - to steer clear of the site. In other words, if a site has this file attached, it can't be archived by wayback machine or googled. This practice is often used with livejournal posts, if someone enables a certain option in their account settings, their posts won't pop up in google searches and will be invisible to wayback machine.

As for why links work one day and not the next, there can be several reasons. Some of which:

1) there might be a problem with the website itself. Every now and then Livejournal has some problems and pages load improperly or don't load at all.

2) you're connected via random proxy and it might be blocked

3) problems with your ISP (internet service provider). If a link that worked yesterday doesn't work today, go to google.com and google anything whatsoever ('Michael Jackson', 'how to raise a pet dinosaur' or 'why does God hate America' or anything else). If googling doesn't work, your internet connection is off or malfunctioning. Or you're currently in China. Sometimes if your internet connection is spotty (like during peak hours), you can access fast sites like google, but others (like, say, youtube or livejournal) take too long to load and the connection is aborted.

4) you might have a virus

5) problems with browser cache (I won't go into detail here, because frankly I'm not sure how to solve those problems, especially on browsers other than firefox)

6) other reasons

[amelialourdes.livejournal.com]

You're so smart.

[wouldbedorothy.livejournal.com]

Are you having trouble with the original links to fics and then Wayback isn't helping, or are links that you only had through Wayback no longer working?

Rin sounds like she knows what she's talking about; I do not, lol. But I know Wayback links tend to be... precarious. So I would suggest saving copies of any fics like that you don't want to lose in future. Might any of these be in our fic folder?

Your post is fine. We're not just fic search, but this is about fic-searching, anyway. If you could make your subject line more specific to your issue, that would be great :)

[amydambly.livejournal.com]

If I click on the link to the side on this page, it will take me with no trouble to the main The Wayback Machine page but none of the links to stories I had saved that worked at one time work now. I wouldn't be surprised if I had a virus. Every PC I've ever owned has been destroyed beyond repair by them so I seem to be a magnet for them. This computer I just had in for work 3 months ago and they found 4 viruses that McAfee didn't. Oh well...

[rinmonsterer.livejournal.com]

I suggest you uninstall McAfee, as it is the most unreliable anti-virus. It doesn't detect a lot of threats and sometimes tells you to remove your legitimate system files, which might crash your computer. If you don't want to spend money on an antivirus (if you do, I'd suggest Kaspersky, maybe?) and have windows 7 or vista, get Microsoft Security Essentials (http://windows.microsoft.com/en-US/windows/products/security-essentials). And if you don't already, use firefox and, which is important! install the adblock addon (https://addons.mozilla.org/en-US/firefox/addon/adblock-plus/). If you want some help with those things, feel free to PM me. I'm not a specialist, but I might be able to point you in the right direction :)

[amydambly.livejournal.com]

Yeah, I've not heard good things about McAfee...it came with the computer and then, after the free 15 months subscription, I just renewed it. I have 3 months left and figured I'd wait it out but now I'm thinking that was dumb. Even the guy who fixed it before said McAfee was crap but since I had Spyware doctor as a back-up of sort, he thought it'd be ok to wait too. I'm in safe mode now because even IF windows decides to load and I manage to get online, it freezes. Weird because it was fine all evening yesterday. Thanks for all you input!!