Mon . 19 Apr 2019

Bigtable

bigtable.net, www.bigtable.com
Bigtable is a compressed, high performance, and proprietary data storage system built on Google File System, Chubby Lock Service, SSTable log-structured storage like LevelDB and a few other Google technologies On May 6, 2015, a public version of Bigtable was made available as a service Bigtable also underlies Google Cloud Datastore, which is available as a part of the Google Cloud Platform

Contents

  • 1 History
  • 2 Design
  • 3 Other similar software
  • 4 See also
  • 5 References
  • 6 Bibliography
  • 7 External links

History

Bigtable development began in 2004 and is now used by a number of Google applications, such as web indexing, MapReduce, which is often used for generating and modifying data stored in Bigtable, Google Maps, Google Book Search, "My Search History", Google Earth, Bloggercom, Google Code hosting, YouTube, and Gmail Google's reasons for developing its own database include scalability and better control of performance characteristics

Google's Spanner RDBMS is layered on an implementation of Bigtable with a Paxos group for two-phase commits to each table Google F1 was built using Spanner to replace an implementation based on MySQL

Design

Bigtable maps two arbitrary string values row key and column key and timestamp hence three-dimensional mapping into an associated arbitrary byte array It is not a relational database and can be better defined as a sparse, distributed multi-dimensional sorted map:1 Bigtable is designed to scale into the petabyte range across "hundreds or thousands of machines, and to make it easy to add more machines the system and automatically start taking advantage of those resources without any reconfiguration"

Each table has multiple dimensions one of which is a field for time, allowing for versioning and garbage collection Tables are optimized for Google File System GFS by being split into multiple tablets – segments of the table are split along a row chosen such that the tablet will be ~200 megabytes in size When sizes threaten to grow beyond a specified limit, the tablets are compressed using the algorithm BMDiff and the Zippy compression algorithm publicly known and open-sourced as Snappy, which is a less space-optimal variation of LZ77 but more efficient in terms of computing time The locations in the GFS of tablets are recorded as database entries in multiple special tablets, which are called "META1" tablets META1 tablets are found by querying the single "META0" tablet, which typically resides on a server of its own since it is often queried by clients as to the location of the "META1" tablet which itself has the answer to the question of where the actual data is located Like GFS's master server, the META0 server is not generally a bottleneck since the processor time and bandwidth necessary to discover and transmit META1 locations is minimal and clients aggressively cache locations to minimize queries

Other similar software

  • Apache Accumulo — built on top of Hadoop, ZooKeeper, and Thrift Has cell-level access labels and a server-side programming mechanism Written in Java
  • Apache Cassandra — brings together Dynamo's fully distributed design and Bigtable's data model Written in Java
  • Apache HBase — Provides Bigtable-like support on the Hadoop Core Has cell-level access labels and a server-side programming mechanism too Written in Java
  • Hypertable — Hypertable is designed to manage the storage and processing of information on a large cluster of commodity servers Written in C++
  • "KDI", Bluefish, GitHub  — Kosmix attempt to make a Bigtable clone Written in C++
  • LevelDB — Google's embedded key/value store that uses similar design concepts as the Bigtable tablet

See also

  • Amazon SimpleDB
  • Big data
  • Distributed data stores, an overview
  • Dynamo storage system
  • Column-oriented DBMS
  • Hadoop

References

  1. ^ "Announcing Google Cloud Bigtable: The same database that powers Google Search, Gmail and Analytics is now available on Google Cloud Platform" Google Blog May 6, 2015 Retrieved September 21, 2016 
  2. ^ http://googledevelopersblogspotcom/2013/05/get-started-with-google-cloud-datastorehtml
  3. ^ Kumar, Aswini, Whitchcock, Andrew, ed, Google's BigTable, First an overview BigTable has been in development since early 2004 and has been in active use for about eight months about February 2005 
  4. ^ Chang, Fay; Dean, Jeffrey; Ghemawat, Sanjay; Hsieh, Wilson C; Wallach, Deborah A; Burrows, Michael ‘Mike’; Chandra, Tushar; Fikes, Andrew; Gruber, Robert E 2006, "Bigtable: A Distributed Storage System for Structured Data", download ebook PDF, Google 
  5. ^ Chang et al 2006, p 3: ‘Bigtable can be used with MapReduce, a framework for running large-scale parallel computations developed at Google We have written a set of wrappers that allow a Bigtable to be used both as an input source and as an output target for MapReduce jobs’
  6. ^ Whitchcock, Andrew, Google's BigTable, There are currently around 100 cells for services such as Print, Search History, Maps, and Orkut 
  7. ^ Cordes, Kyle 2007-07-12, YouTube Scalability talk, Their new solution for thumbnails is to use Google’s BigTable, which provides high performance for a large number of rows, fault tolerance, caching, etc This is a nice and rare example of actual synergy in an acquisition 
  8. ^ "How Entities and Indexes are Stored", Google App Engine, Google Code 
  9. ^ Chang et al 2006, Conclusion: ‘We have described Bigtable, a distributed system for storing structured data at Google Our users like the performance and high availability provided by the Bigtable implementation, and that they can scale the capacity of their clusters by simply adding more machines to the system as their resource demands change over time Finally, we have found that there are significant advantages to building our own storage solution at Google We have gotten a substantial amount of flexibility from designing our own data model for Bigtable’
  10. ^ Shute, Jeffrey ‘Jeff’; Oancea, Mircea; Ellner, Stephan; Handy, Benjamin ‘Ben’; Rollins, Eric; Samwel, Bart; Vingralek, Radek; Whipkey, Chad; Chen, Xin; Jegerlehner, Beat; Littlefield, Kyle; Tong, Phoenix 2012, "Summary; F1 — the Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business", Research presentation, Sigmod: Google, p 19, We've moved a large and critical application suite from MySQL to F1 
  11. ^ Chang et al 2006
  12. ^ "Google File System and BigTable", Radar World Wide Web log, Database War Stories 7, O’Reilly, May 2006 
  13. ^ "Google Bigtable, Compression, Zippy and BMDiff" 2008-10-12 Archived from the original on 1 May 2013 Retrieved 14 April 2015 
  14. ^ McIlroy, Bentley Data compression using long common strings DCC '99 IEEE 
  15. ^ "Google's Bigtable", Outer court Weblog, 2005-10-23 
  16. ^ "Snappy", Code project, Google 
  17. ^ "Background; HBase", Hadoop Core wiki, Apache 
  18. ^ "About", Hyper table 
  19. ^ "Leveldb file layout and compactions", Code, Google 

Bibliography

  • Chang, Fay; Dean, Jeffrey; Ghemawat, Sanjay; Hsieh, Wilson C; Wallach, Deborah A; Burrows, Michael ‘Mike’; Chandra, Tushar; Fikes, Andrew; Gruber, Robert E 2006, "Bigtable: A Distributed Storage System for Structured Data", download ebook PDF, Google 

External links

  • BigTable: A Distributed Structured Storage System, Washington  Video, Google 
    • UWTV video 
    • Witchcock, Andrew, Google's BigTable notes on the official presentation 
  • Carr, David F 2006-07-06, "How Google Works", Baseline 
  • "Is the Relational Database Doomed", Read-write web 

big table, big table farm winery, big table google, big table lamps, big table media, big table paper, big table publishing, big tablet, bigtable.net, www.bigtable.com


Bigtable Information about

Bigtable


  • user icon

    Bigtable beatiful post thanks!

    29.10.2014


Bigtable
Bigtable
Bigtable viewing the topic.
Bigtable what, Bigtable who, Bigtable explanation

There are excerpts from wikipedia on this article and video

Random Posts

Modern philosophy

Modern philosophy

Modern philosophy is a branch of philosophy that originated in Western Europe in the 17th century, a...
Tim Shadbolt

Tim Shadbolt

Timothy Richard "Tim" Shadbolt born 19 February 1947 is a New Zealand politician He is the Mayor of ...
HK Express

HK Express

Andrew Cowen Deputy CEO Website wwwhkexpresscom HK Express Traditional Chinese 香港快運航空...
List of shrinking cities in the United States

List of shrinking cities in the United States

The following municipalities in the United States have lost at least 20% of their population, from a...