Notes for BSc CSIT

 Extent Ans: An extent is a logical unit of database storage space allocation made up of a number of contiguous data blocks. One or more extents in turn make up a segment. When the existing space in a segment is completely used, Oracle allocates a new extent for the segment. When Extents Are Allocated When you create a table, Oracle allocates to the table's data segment an initial extent of a specified number of data blocks. Although no rows have been inserted yet, the Oracle data blocks that correspond to the initial extent are reserved for that table's rows. If the data blocks of a segment's initial extent become full and more space is required to hold new data, Oracle automatically allocates an incremental extent for that segment. An incremental extent is a subsequent extent of the same or greater size than the previously allocated extent in that segment. For maintenance purposes, the header block of each segment contains a directory of the extents in that segment. Tempo

Multimedia database A multimedia database management system (MM-DBMS) is a framework that manages different types of data potentially represented in a wide diversity of formats on a wide array of media sources. Benefits of Multimedia database:  They support multiple formats of data (text, audio, video, etc)  Data independence: Separate the database and the management from the application program.  They provide access and authorization control.  Query support: Multimedia databases should have the ability to uniformly Query data (media data, Textual data) represented in different formats and have the ability to simultaneously query different media sources and conduct classical database operations across them. Multimedia databases provide features that allow users to store and query different types of multimedia information, which includes images (such as photos or drawings), video clips (such as movies, newsreels, or home videos), audio clips (such as songs, phone messages, or speeches

Multimedia Database Characteristics  A  multimedia database contains various data types such as text,  images,  graphic objects (including pictures,  drawings, and illustrations),  animation sequences,  video, and audio.  Additionally, Kalipsiz  summarized some characteristics of multimedia data below:   • Lack of structure:  Multimedia data often are not quite structured;  therefore,  standard indexing and/or content-based search and retrieval may not be available.   • Temporality: Different multimedia data types have different requirements. For example, some multimedia data types such as video, audio, and animation sequences have temporal requirements that have implications on their storage, manipulation, and presentation, but images, video, and graphics data have spatial constraints in terms of their content.   • Massive Volume:  Usually,  the data size of multimedia is large such as video; therefore, multimedia data often require a large storage device.   • Logistics:  Non-standa

Multimedia Database  A multimedia database is a collection of related multimedia data. Common multimedia data types that can be found in a multimedia database include the following:    • Text  • Graphics: drawing, sketches, and illustrations  • Images: color and black & white pictures, photographs, maps, and paintings  • Animation sequences: animated images or graphic objects  • Video:  a  sequence  of images  (frames),  typically  recording  a  real-life  event  and  usually produced by a video recorder  • Audio: generated from an aural recording device  • Composite multimedia: a combination of two or more of the above data type   Multimedia Database Applications  Multimedia database applications are different from traditional database applications in the structure of the multimedia objects and media where the multimedia objects are stored and retrieved.  Multimedia data is diverse with different characteristics.  Because of the audio-visual nature of multimedia data types,  they

 Multimedia Database  A Multimedia database is the collection of interrelated multimedia data that includes text, graphics (sketches, drawings), images, animations, video, audio, etc., and has vast amounts of multisource multimedia data. The framework that manages different types of multimedia data which can be stored, delivered, and utilized in different ways is known as a multimedia database management system. Challenges to multimedia databases Modeling : Work in this area can improve the database versus information retrieval techniques. Design: The physical, conceptual, and logical design of multimedia databases is not addressed entirely leading to performance and tuning issues. Storage: The storage of databases on a standard disc can lead to problems like representation, mapping to disc hierarchies, compression, etc. Performance: Audio-video synchronization and audio playback applications are where physical limitations dominate. Parallel processing can reduce these problems, but th

 There are many applications of the temporal database are f(reservation systems, scientific databases, company databases, university databases,s, etc.) where some aspect of time is needed to maintain the information in a database  Insurance, w here claims and accident histories are required as well as information about the times when insurance policies are in effect;  Reservation systems in general (hotel, airline, car rental, train, and so on), where information on the dates and times when reservations are in effect is required;  Scientific databases, where data collected from experiments include the time when each data is measured; and so on In the COMPANY database , we may wish to keep SALARY, JOB, and PROJECT histories on each employee  In the UNIVERSITY database, time is already included in the SEMESTER and YEAR of each SECTION of a COURSE, the grade history of a STUDENT, and the information on research grants.  In fact, it is realistic to conclude that the majority of database

 Information retrieval  Information retrieval is the process of retrieving documents from a collection in response to a query (or a search request) by a user. Information Retrieval is the activity of obtaining material that can usually be documented in an unstructured nature i.e. usually text which satisfies an information need from within large collections which is stored on computers. For example, Information Retrieval can be when a user enters a query into the system. Nowadays hundreds of millions of people engage in IR every day when they use web search engines. Information Retrieval is believed to be the dominant form of Information access. The IR system assists the users in finding the information they require but it does not explicitly return the answers to the question. It notifies regarding the existence and location of documents that might consist of the required information. Information retrieval also extends support to users in browsing or filtering document collection or p

 Deductive Database  A deductive database is a database system that can make deductions (i.e. conclude additional facts) based on rules and facts stored in the (deductive) database. Datalog is the language typically used to specify facts, rules, and queries in deductive databases. A deductive database can be defined as an advanced database augmented with an inference system. Database + Inference = Deductive database Deductive databases A deductive database system is a database system that can make deductions (ie: conclude additional facts) based on rules and facts stored in the (deductive) database. Datalog is the language typically used to specify facts, rules, and queries in deductive databases. Deductive databases have grown out of the desire to combine logic programming with relational databases to construct systems that support a powerful formalism and are still fast and able to deal with very large datasets. Deductive databases are more expressive than relational databases but le

Trigger: A trigger is a stored procedure in a database that automatically invokes whenever a special event in the database occurs. For example, a trigger can be invoked when a row is inserted into a specified table or when certain table columns are being updated. OR,  A trigger is a procedure that is automatically invoked by the DBMS in response to changes to the database and is specified by the database administrator (DBA). A database with a set of associated triggers is generally called an active database. These databases are very difficult to be maintained because of the complexity that arises in understanding the effect of these triggers. In such a database, DBMS initially verifies whether the particular trigger specified in the statement that modifies the database) is activated or not, prior to executing the statement. If the trigger is active then DBMS executes the condition part and then executes the action part only if the specified condition is evaluated to true. It is possibl

  Temporal Database Concepts. A temporal database stores data relating to time instances. It offers temporal data types and stores information relating to past, present, and future times. A Temporal Database is a database with built-in support for handling time-sensitive data. Usually, databases store information only about the current state, and not about past states.  For example in an employee database, if the address or salary of a particular person changes, the database gets updated, and the old value is no longer there. However, for many applications, it is important to maintain the past or historical values and the time at which the data was updated. That is, the knowledge of evolution is required. That is where temporal databases are useful.  It stores information about the past, present, and future. Any data that is time-dependent is called temporal data and these are stored in temporal databases. Temporal Databases store information about states of the real world across time.

Active Databases   A trigger is a procedure that is automatically invoked by the DBMS in response to changes to the database and is specified by the database administrator (DBA). A database with a set of associated triggers is generally called an active database.  Generalized Model for Active Databases  The model that has been used to specify active database rules is referred to as the Event-Condition-Action (ECA) model. A rule in the ECA model has three components: 1. The event(s) that triggers the rule: These events are usually database update operations that are explicitly applied to the database. However, in the general model, they could also be temporal events 2 or other kinds of external events. 2. The condition that determines whether the rule action should be executed: Once the triggering event has occurred, an optional condition may be evaluated. If no condition is specified, the action will be executed once the event occurs. If a condition is specified, it is first evaluate

 Need for a Temporal DBMS  It is difficult to identify applications not needing management of temporal data  These applications would benefit from built-in temporal support in the DBMS • More efficient application development • Potential increase in performance  Temporal DBMS: Provide mechanisms to store and manipulate time-varying information. OR, We need a temporal database because:- A temporal database stores data relating to time instances. It offers temporal data types and stores information relating to past, present, and future times. A Temporal Database is a database with built-in support for handling time-sensitive data. Usually, databases store information only about the current state, and not about past states. For example in an employee database, if the address or salary of a particular person changes, the database gets updated, and the old value is no longer there.  However, for many applications, it is important to maintain the past or historical values and the time at whi

Application of Active Database  Data monitoring activities such as CIM, Telecommunications Network Management, and Program trading. Medical and Financial Decision Support Systems can greatly benefit from integration with an active database. Production control, e.g., power plants. Maintenance tasks, e.g., inventory control. Financial applications, e.g., stock & bond trading. Telecommunication and network management. Air traffic control. Computer Integrated Manufacturing (CIM) Statistics gathering and authorization tools. Spatial databases differ from regular databases because Common database systems use indexes for a faster and more efficient search and access to data. This index, however, is not fit for spatial queries. Instead, spatial databases use something like a unique index called a spatial index to speed up database performance. Spatial indexing is very much required because a system should be able to retrieve data from a large collection of objects without really searching

 The uses of an active database are as follows:- It possesses all the concepts of a conventional database i.e. data modeling facilities, query language, multi-user access, recovery, etc. It supports all the functions of a traditional database, including data definition, data manipulation, storage management, transaction management, concurrency control, and crash recovery. An active database supports the definition and management of ECA rules. An active database must detect event occurrences. An active database must be able to evaluate conditions and execute actions. An active database includes an event-driven architecture (often in the form of ECA rules) An active database must be able to evaluate conditions and execute actions. This requirement means that it has to implement rule execution.  Active Database A trigger is a procedure that is automatically invoked by the DBMS in response to changes to the database and is specified by the database administrator (DBA).A database with a set

 Features of Hadoop  1. Open Source: Hadoop is open-source, which means it is free to use. Since it is an open-source project the source code is available online for anyone to understand it or make some modifications as per their industry requirement. 2. Highly Scalable Cluster: Hadoop is a highly scalable model. A large amount of data is divided into multiple inexpensive machines in a cluster which is processed parallelly. the number of these machines or nodes can be increased or decreased as per the enterprise’s requirements. In traditional RDBMS(Relational DataBase Management System) the systems can not be scaled to approach large amounts of data. 3. Fault Tolerance is Available: Hadoop uses commodity hardware(inexpensive systems) which can be crashed at any moment. In Hadoop data is replicated on various DataNodes in a Hadoop cluster which ensures the availability of data if somehow any of your systems got crashed. You can read all of the data from a single machine if this machine

Businesses are using Big Data for competitive advantage because:-  According to the Mckinsey Global Institute, there are 5 ways how Big Data creates value (McKinsey Global Institute, 2011):   It can create transparency by being more widely available to the new potential.   It enables companies to set up experiments. For example experiments for process changes, they can create and analyze large amounts of data from these experiments to identify possible performance improvements.   Big Data can be used to create a more detailed segmentation of customers to customize actions and prepare specific services.  Analysis of Big Data can support human decision-making by pointing to hidden correlations or some hidden risks. An example can be a risk or fraud analysis engine for insurance companies. Low decision-making can be even automated to those engines in some cases. Data can also enable new business models, products, and services or can improve the existing ones. Data about how products and s

Cloud computing and big data are complementary to each other. Rapid growth in big data is regarded as a problem. Clouds are evolving and providing solutions for the appropriate environment of big data while traditional storage cannot meet the requirements for dealing with big data, in addition to the need for data exchange between various distributed storage locations. Cloud computing provides solutions and addresses problems with big data. The cloud computing environment is expanding to be able to absorb big amounts of data as it follows the policy of data splitting, that is, to store data in more than one location or availability area. Cloud computing environments are built for general purpose workloads and resource pooling is used to provide flexibility on demand. Therefore, the cloud computing environment seems to be well suited for big data. OR,  Big Data is the large data set collected from large network-based systems. The Cloud is the location where this data is processed and ac

 Hadoop is an open-source framework that allows to storage and process of big data in a distributed environment across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Hadoop is written in Java and is not OLAP (online analytical processing). It is used for batch/offline processing. It is being used by Facebook, Yahoo, Google, Twitter, LinkedIn, and many more. Moreover, it can be scaled up just by adding nodes in the cluster. Hadoop Architecture The Hadoop architecture is a package of the file system, MapReduce engine, and the HDFS (Hadoop Distributed File System). The MapReduce engine can be MapReduce/MR1 or YARN/MR2. A Hadoop cluster consists of a single master and multiple slave nodes. The master node includes Job Tracker, Task Tracker, NameNode, and DataNode whereas the slave node includes DataNode and TaskTracker. Hadoop Distributed File System The Hadoop Dist

 THE CAP THEOREM CAP stands for Consistency, Availability, and Partitioning. It is very important to understand the limitations of the NoSQL database. NoSQL cannot provide consistency and high availability together. This was first expressed by Eric Brewer in CAP Theorem. CAP theorem or Eric Brewer's theorem states that we can only achieve at most two out of three guarantees for a database: Consistency, Availability, and Partition Tolerance. Consistency Consistency is all about data consistency, or in other words, making sure that within a distributed environment, every node of the database has exactly the same information at any given time. Imagine having two nodes with purchase orders from your eCommerce site. If there is no data and Bigdata 209 consistency amongst them and they're acting as a unique cluster, the moment your client app queries the outdated node, it might show you missing transactions. And if the code is not only showing, but also making calculations based on t

 CHARACTERISTICS RELATED TO DISTRIBUTED DATABASES AND DISTRIBUTED SYSTEMS  1.  Scalability:  horizontal scalability: adding more nodes for data storage and processing as the volume of data grows.  Vertical scalability: expanding the storage and computing power of existing nodes.  In NoSQL systems, horizontal scalability is employed while the system is operational, so techniques for distributing the existing data among new nodes without interrupting system operation are necessary.  2. Availability, Replication, and Eventual Consistency:  Data is replicated over two or more nodes in a transparent manner.  An update must be applied to every copy of the replicated data items.   Eventual consistency: is a consistency model used in distributed computing to achieve high availability that informally guarantees that, if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value.  3. Replication Models:  Master-slave replication: requ