Notes for BSc CSIT

PARALLEL COMPUTING Parallel computing is a sort of computer architecture in which many processors simultaneously execute or process an application or calculation. Parallel computing aids in the performance of big calculations by splitting the workload across several processors, all of which work on the computation at the same time. The majority of supercomputers run using parallel computing methods. Parallel processing is another name for parallel computing. Parallel processing is typically used in operating environments/scenarios that need large computing or processing capability. Parallel computing's primary goal is to enhance available computing power for quicker application processing or job resolution. Parallel computing infrastructure is often hosted in a single facility where multiple processors are deployed in a server rack or independent servers are linked together. The application server provides a calculation or processing request that is broken down into little

 MULTI-TENANT CLOUD A multi-tenant cloud is a cloud computing architecture that enables clients to share computer resources in either the public or private cloud.  Each tenant's data is segregated and hidden from other residents. Users in a multi-tenant cloud system have their area to store their projects and data. Each segment of a multi Tenant cloud network comprises sophisticated permissions to provide each user access to just their stored formation while also protecting them from other cloud tenants. Each tenant's data is unavailable to all her tenants inside the cloud architecture and may only be accessed with the cloud provider's rights. Customers, or tenants, in a private cloud, might be various individuals or groups inside a single firm, but on a public cloud, completely separate enterprises can securely share their server space. The multi-tenancy approach is used by the majority of public cloud providers. It enables them to run servers with single lances, which sav

 AMAZON SIMPLEDB Amazon SimpleDB is a highly available NoSQL data storage that relieves database administrators of their duties. Developers just use web service requests to save and access data objects, and Amazon SimpleDB handles the rest. Amazon SimpleDB, unlike relational databases, is designed to provide high availability and flexibility with minimal or no administration overhead. Amazon SimpleDB automatically builds and manages numerous globally dispersed duplicates of your data behind the scenes to provide high availability and data durability. The service only costs you for the resources used in storing your data and delivering your requests. You may alter your data model on the fly, and data is indexed for you automatically. You can concentrate on application development without having to worry about infrastructure provisioning, high availability, software maintenance, schema, and index management, and performance optimization using Amazon SimpleDB.  This SimpleDB service provi

 GOOGLE CLOUD DATASTORE Google Cloud Datastore is a Google Cloud Platform service that provides a highly scalable, fully managed NoSQL database. Cloud Datastore is based on Bigtable and Megastore technologies from Google. Cloud Datastore is a NoSQL document database designed for online and mobile applications that require automated scalability, excellent performance, and simplicity of development. Features of Google Cloud Datastore include: Atomic transactions.  Cloud Datastore can perform a series of operations in which they succeed. High availability of reads and writes.  Cloud Datastore is hosted in Google data centers, which employ redundancy to reduce the effects of single points of failure. Massive scalability with high performance.  To manage scalability automatically, Cloud Datastore has a distributed design. Cloud Datastore employs a combination of indexes and query limitations to ensure that your queries grow with the size of your result set rather than the size of your data

 AMAZON DYNAMO Amazon offers DynamoDB, a fully managed proprietary NoSQL database service that supports key-value and document data formats, as part of the Amazon Web Services portfolio. DynamoDB provides a comparable data architecture to Dynamo and gets its name from it, but it has a distinct underlying implementation. DynamoDB employs synchronous replication over many data centers for high durability and availability, and it has a multi-master design that requires the client to resolve version conflicts. Amazon DynamoDB is a key-value and document database that promises performance in single-digit milliseconds at any size. It is a fully managed, multiregional, multi-master database for internet-scale applications that has built-in security, backup and restore, and in-memory caching. DynamoDB can handle more than 10 trillion requests per day, with peaks of more than 20 million requests per second. Many of the world's fastest-growing companies, like Lyft, Airbnb, and Redfin, as wel

Differences between HDFS and HBase. HDFS  HDFS is a distributed file system suitable for storing large files. HDFS does not support fast individual record lookups. It provides high latency batch processing; no concept of batch processing. It provides only sequential access to data. HBase HBase is a database built on top of the HDFS. HBase provides fast lookups for larger tables. It provides low latency access to single rows from billions of records (Random access).  HBase internally uses Hash tables and provides random . Difference between HBase and RDBMS. HBase HBase is schema-less, it does not have the concept of fixed columns schema; defines only column families. It is built for wide tables. HBase is horizontally scalable. No transactions are there in HBase. It has de-normalized data. It is good for semi-structured as well as structured data. RDBMS An RDBMS is governed by its schema, which describes the whole structure of tables. It is thin and built for small tables. Hard to scale.

HBase HBase is a column-oriented distributed database developed on top of the Hadoop file system. It is an open-source project that may be scaled horizontally. HBase is a data architecture comparable to Google's big table that is meant to allow fast random access to massive volumes of structured data. It makes use of the Hadoop File System's fault tolerance (HDFS). Apache HBase is a Hadoop -based distributed, scalable NoSqlb Big data storage.HBase is capable of hosting very large tables-billions of rows and millions of columns-and of providing real-time, random read/write access to Hadoop data. HBase is a multi-column data store inspired by Google Bigtable, a database interface to Google's proprietary File System. HBase adds Bigtable-like features to read/write access to Hadoop-compatible file systems like MapR XD. HBase scales linearly over very large datasets and allows for the easy combination of data sources with heterogeneous topologies and schemas.   Features of HBase

 STORAGE MECHANISM IN HBASE HBase is a column-oriented database, with tables ordered by row. Only column families, which are key-value pairs, are defined in the table structure. A table contains many columns families, each of which can include any number of columns. Subsequent column values are saved on the disk in a logical order. A timestamp is associated with each cell value in the table. In a nutshell, in an HBase: The table is a collection of rows. The row is a collection of column families. A Column family is a collection of columns.  The column is a collection of key-value pairs. An example schema of a table in HBase is provided below. COLUMN-ORIENTED AND ROW-ORIENTED DATABASE Column-oriented databases, as opposed to row-oriented databases, store data tables as portions of columns of data. They will have column families. Row-Oriented Database It is suitable for the Online Transaction Process(OLTP). Such databases are designed for a small number of rows and columns. Column-Orient

APACHE HBASE HBase is a column-oriented distributed database developed on top of the Hadoop file system. It is an open-source project that may be scaled horizontally. HBase is a data architecture comparable to Google's big table that is meant to allow fast random access to massive volumes of structured data. It makes use of the Hadoop File System's fault tolerance (HDFS). Apache HBase is a Hadoop -based distributed, scalable NoSqlb Big data storage.HBase is capable of hosting very large tables-billions of rows and millions of columns-and of providing real-time, random read/write access to Hadoop data. HBase is a multi-column data store inspired by Google Bigtable, a database interface to Google's proprietary File System. HBase adds Bigtable-like features to read/write access to Hadoop-compatible file systems like MapR XD. HBase scales linearly over very large datasets and allows for the easy combination of data sources with heterogeneous topologies and schemas.  HBase is a

 GOOGLE BIGTABLE Google Bigtable is a distributed, column-oriented data store developed by Google Inc. to manage massive volumes of structured data related to the company's Internet search and Web services operations. Bigtable was created to enable applications that need large scalability; the technology was meant to be utilized with petabytes of data from its inception. The database was designed to run on clustered servers and has a basic data format described by Google as "a sparse, distributed, permanent multi-dimensional sorted map." The data is placed in order by row key, and the map's indexing is sorted by row, column keys, and timestamps. Algorithms for compression aid in achieving high capacity. BigTable is a petabyte scale, fully managed NoSQL database service designed for big analytical and operational workloads. Google Cloud Bigtable is a NoSQL Big Data database service. It's the same database that underpins many of Google's main services, such as S

 HDFS Architecture The architecture of a Hadoop File System is shown in figure 7.2. Those components of the architecture are described as below: Namenode The namenode is the commodity hardware that contains the GNU/Linux operating system and the namenode software. It is software that can be run on commodity hardware. The system having the namenode acts as the master server and it does the following tasks: •Manages the file system namespace.  •Regulates client's access to files. •It also executes file system operations such as renaming, closing, and opening files and directories. Datanode The datanode is a piece of commodity hardware that runs the GNU/Linux operating system as well as the datanode software. A datanode will exist for each node (Commodity hardware/System) in a cluster. These nodes are in charge of their system's data storage. Datanodes perform read-write operations on the file systems, as per the client's request.  They also perform operations such as block cr

Hadoop File System(HDFS)  The Hadoop File System was created with a distributed file system design. It runs on standard hardware. In contrast to other distributed systems, HDFS is extremely faulted tolerant and built with low-cost hardware. HDFS stores a big quantity of data and makes it easy to access. To store such large amounts of da the files are spread over numerous computers. These files are kept in a redundant form to protect the system from data loss in the event of a breakdown. HDFS also enables parallel processing of applications. Hadoop applications use the Hadoop Distributed File Solution (HDFS) as their primary data storage system. It implements a distributed file system that allows high-performance access to data across highly scalable Hadoop clusters using a NameNode and DataNode architecture. HDFS is an important component of the numerous Hadoop ecosystem technologies since it provides a dependable way of maintaining massive data pools and supporting associated big data

Google File System (GFS)   The Google File System is a scalable distributed file system designed for big data-intensive distributed Applications. It offers fault tolerance while running on low-cost commodity hardware and gives great distributed file systems, its design has been influenced by observations of application workloads and the aggregate performance of a large number of customers. While GFS has many of the same aims as past technical environments, both current and prospective, which represent a significant divergence from Certain preceding file system assumptions. As a result, established options have been reexamined, and Grammatically alternative design points have been explored. Google File System (GFS) is a scalable distributed file system (DFS) designed by Google Inc. to meet Google's growing data processing needs. GFS supports huge networks and linked nodes with fault tolerance, dependability, scalability, availability, and performance. GFS is comprised of storage sys

  RELATIONAL DATABASES A relational database is a form of database that stores and allows access to data elements that are connected. Relational databases are based on the relational model, which is an easy-to-understand method of expressing data in tables. In a relational database, each row in the table is a record with a unique ID called the key. The columns of the table carry data attributes, and each record generally includes a value for each attribute, making it simple to construct links between data points. In a relational database, each table, also known as a relation, includes one or more data categories in columns, also known as attributes. Each row, also known as a record or tuple, includes a unique instance of data, or key, for the columns' stated categories. Each table has a unique primary key that identifies the data in the table. The relationship between tables may then be defined using foreign keys, which are fields in one table that are linked to the primary key of

ACID and Relational Databases Four crucial properties define relational database transactions: atomicity, consistency, isolation, and rability-typically referred to as ACID.  Atomicity defines all the elements that make up a complete database transaction or none. Consistency defines the rules for maintaining data points in a correct state after a transaction. Isolation keeps the effect of a transaction invisible to others until it is committed, to avoid confusion.  Durability ensures that data changes become permanent once the transaction is committed. The Relational Database of the Future: The Self-Driving Database Relational databases have grown better, quicker, stronger, and easier to deal with throughout time. They have, however, become more complicated, and maintaining the database has long been a full-time job. Instead of focusing on designing creative applications that provide value to the company, developers have had to spend the majority of their time on the administrative a

 RELATIONAL DATABASES A relational database is a form of database that stores and allows access to data elements that are connected. Relational databases are based on the relational model, which is an easy-to-understand method of expressing data in tables. In a relational database, each row in the table is a record with a unique ID called the key. The columns of the table carry data attributes, and each record generally includes a value for each attribute, making it simple to construct links between data points. In a relational database, each table, also known as a relation, includes one or more data categories in columns, also known as attributes. Each row, also known as a record or tuple, includes a unique instance of data, or key, for the columns' stated categories. Each table has a unique primary key that identifies the data in the table. The relationship between tables may then be defined using foreign keys, which are fields in one table that are linked to the primary key of a

 WIDELY SEEN SECURITY ISSUES 1. Data breaches:  A data breach might be the primary goal of a targeted attack, or it might be the consequence of a human mistake, application flaws, or inadequate security policies. It might include any material that was not meant for public distribution, such as personal health information, financial information, personally-identifying information, trade secrets, and intellectual property. The cloud-based data of a business may be valuable to many parties for a variety of reasons. The danger of data breaches is not unique to cloud computing, but it remains a top issue for cloud consumers. 2. Insufficient identity, credential, and access management:  Bad actors impersonating genuine users, operators, or developers can access, edit, and delete data; issue control plane and management functions; eavesdrop on data in transit, or distribute harmful software that looks to come from a genuine source. As a result, inadequate identity, credentials, or key managem

  HIGH AVAILABILITY AND FAULT TOLERANCE An effective IT infrastructure must function even in the event of a rare network loss, device failure, or power loss. When the system fails, one or more of the three major availability techniques will kick in: high availability, fault tolerance, and/or disaster recovery. While each of these infrastructure design solutions contributes to the availability of your key applications and data, they do not fulfill the same goal. Simply because you run a High Availability infrastructure does not mean you need not set up a disaster recovery site and doing so risks disaster. HIGH AVAILABILITY A High Availability system is meant to be up and running 99.99 percent of the time, or as close to it as feasible. Typically, this entails creating a failover system capable of handling the same workloads as the original system. HA works in a virtualized environment by generating a pool of virtual computers and related resources inside a cluster. When one of the hosts

  HIGH AVAILABILITY AND FAULT TOLERANCE An effective IT infrastructure must function even in the event of a rare network loss, device failure, or power loss. When the system fails, one or more of the three major availability techniques will kick in: high availability, fault tolerance, and/or disaster recovery. While each of these infrastructure design solutions contributes to the availability of your key applications and data, they do not fulfill the same goal. Simply because you run a High Availability infrastructure does not mean you need not set up a disaster recovery site and doing so risks disaster. HIGH AVAILABILITY A High Availability system is meant to be up and running 99.99 percent of the time, or as close to it as feasible. Typically, this entails creating a failover system capable of handling the same workloads as the original system. HA works in a virtualized environment by generating a pool of virtual computers and related resources inside a cluster. When one of the hosts

  CLOUD SECURITY CHALLENGES AND RISKS  Although virtualization and cloud computing can help businesses achieve more by breaking down the physical barriers that exist between an IT infrastructure and its customers, increased security dangers must be faced to fully benefit from this new computing paradigm. This is especially true for SaaS providers. Some security issues merit further consideration. For example, in the cloud, you lose some control over assets, thus your security approach must be re-evaluated. Enterprise security is only as good as its most untrustworthy partner, department, or vendor. Can you put your data in the hands of your service provider? Physical security is lost while using the cloud approach You share computer resources with other businesses in a public cloud. You have no information or control over where the resources run in a common pool outside the organization. Exposing your data in a shared environment with other businesses may provide the authorities with &