Tuesday, 13 March 2012

What lies beneath the Index...

When regular content becomes “too mainstream”.

We know it – everyone uses the Internet. Facebook, YouTube, Twitter are just a few websites people access every day. The time spent on these websites sometimes may go up to many hours on a single day. But that’s exactly where usual user’s concert reaches its end. Truth is: Google does not have all the answers, Facebook doesn’t know all the people and 4chan is not the only website that may scar you for life.
Behind the cover hides what’s sometimes called “Invisible Internet” also known as the Deep Web. So, first of all...

What is the Deep Web?

Good question! For short, the Deep Web consists of non-indexed web pages. In other words, the regular web search engines, such as Google, Yahoo cannot use their bots (crawlers) to access those web pages because of some reasons.
This video gives a brief explanation of what's the deep web.

So what does this Deep Web consist of then??

Okay, so what do you usually see on Facebook?Twitter? Or in summary - all the surface web? Well, it's nothing more than comments, discussions, blog posts, bookmarks, quotations and that's pretty much it.
Now, Deep Web has something more. For example, if you wanted to start a new business, the answer on how to do it is there... but you won't find it! Not really.
The Deep Web holds data created by databases (Access, Oracle, MySQL, SQL Server), non-text multimedia files, such as PDFs, images, software, etc., some content from subscription-based websites. Doesn't look impressive? The information held on databases has no rules, it could be absolutely anything, starting with laws and ending with full articles.

Okay, it's cool. But how do I get ALL this information then?

- Try to figure out the information that might be stored in the database and search for it.
- Use vertical search engines to access the content.

Mining the deep web is a subtle thing and it makes it more interesting at least 10 times.

So, thank you for reading and good luck exploring what lies down there.

Related Posts Plugin for WordPress, Blogger...