Showing posts with label Big Data. Show all posts
Showing posts with label Big Data. Show all posts

Wednesday, November 4, 2015

Big Data : Spark

http://www.computing.co.uk/ctg/analysis/2432898/what-is-spark-six-reasons-why-cios-should-find-out-and-one-why-they-shouldnt

Thursday, September 11, 2014

Big Data Talks

http://www.kdnuggets.com/2014/09/most-viewed-big-data-talks-videolectures.html

http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_innovation 

Mining of massive datasets

http://www.mmds.org/

https://www.linkedin.com/pulse/article/20140918162814-111366377-let-s-get-nerdy-data-analytics-for-business-leaders-explained

http://simplystatistics.org/2014/05/22/10-things-statistics-taught-us-about-big-data-analysis/

http://www.datasciencecentral.com/profiles/blogs/data-science-cheat-sheet

www.coreservlets.com/hadoop-tutorial/

http://www.stratapps.net/intro-hive.php

Monday, June 16, 2014

Big Data Computations Using Elastic Data Processing in OpenStack Cloud

Sahara architecture provides a scalable data processing stack.
  • provision and operate Hadoop clusters
  • schedule and operate Hadoop jobs

https://www.youtube.com/watch?v=-5i_mFwuYKw



http://docs.openstack.org/developer/sahara/architecture.html



Slides can be found here :
http://www.slideshare.net/lukjanovsv/pdf-34637468




Friday, June 13, 2014

Overview of the Lambda Architecture

Source :
http://www.mapr.com/fr/developercentral/lambda-architecture

The Lambda Architecture as seen in the picture has three major components.
  1. Batch layer that provides the following functionality
    1. managing the master dataset, an immutable, append-only set of raw data
    2. pre-computing arbitrary query functions, called batch views.
  2. Serving layer—This layer indexes the batch views so that they can be queried in ad hoc with low latency.
  3. Speed layer—This layer accommodates all requests that are subject to low latency requirements. Using fast and incremental algorithms, the speed layer deals with recent data only.
    Each of these layers can be realized using various big data technologies. For instance, the batch layer datasets can be in a distributed filesystem, while MapReduce can be used to create batch views that can be fed to the serving layer. The serving layer can be implemented using NoSQL technologies such as HBase, while querying can be implemented by technologies such as Apache Drill or Impala. Finally, the speed layer can be realized with data streaming technologies such as Apache Storm or Spark Streaming.