Meta Store in APACHE HIVE

In Hadoop eco-system Hive component processing all the structure information of the various tables and partitions in the warehouse including column ,column type information to the  necessary to read write data and the corresponding HDFS files where data is stored. That is the central repository of Hive metadata.

Here mainly Metadata is the internal database of hive which is responsible for managing metadata information.

 

Meta data which metastore stores contains things like:

1.Ids of database

2.Ids of Tables and index

3.Time of creation of index

4.Time of creation of tables

5.Input format used for tables

6.Output format used for tables

 

Hive majorly Three modes of Metastore

1.Embeded Metastore

2.Local Metastore

3.Remote Metastore

1.Embedded Metastore:

Embedded metastore runs in the same JVM as the Hive service and contains an embedded Derby database instance backed by the local disk it is called the embedded metastore.

Default configuration for Embedded Metastore

below are the metastore configuration details in “hive-site.xml”

 


<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:derby: ;databaseName=/var/lib/hive/metastore/metastore_db;create=true</value>

<descripton>JDBC connect string for a Embedded metastore</description>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>org.apache.derby.jdbc.EmbededDriver</value>

<descripton>Driver class name  for a Embedded metastore</description>

</property>

 

2. Local Metastore:

It supports for multiple users to use a standalone database. Configuration same as local metastore but connect to a database running in a separate process.

If we use MySQL database as the local metastore, then we need some configuration in “hive-site.xml”

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://host/dbname?create DAtabaseIfnotExist=trueue</value>

<descripton>JDBC connect string for a JDBC metastore</description>

</property>

<property>


<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.ConnectionDriverName</value>

<descripton>Driver class name  for a JDBCmetastore</description>

</property>

 

3.Remote Metastore: 

In remoter metastore, where one or more metastore servers run in separate processes to Hive service.

This metastore better manageability and security.

 

Leave a Reply

Your email address will not be published. Required fields are marked *