Fusioninsight HD6.5集成Impala-大数据-Yjc Blog

Fusioninsight HD6.5没有impala组件，要在FI中集成Impala，可以通过以下步骤：
1. 对Apache Impala 3.4进行编译
2. 构建Impala独立安装所需的目录、启动文件
3. 在FI管理平台中创建所需用户及用户授权

一、编译Impala

1. 环境要求

最小内存：4GB，低于4G编译将失败
操作系统：选择和FI集群一样的操作系统即可
硬盘：20G

2. 下载Apache Impala3.4

# 克隆impala 3.4
git clone --single-branch --branch 3.4.0 https://github.com/apache/impala.git impala-3.4

# 打上 IMPALA-9815 的commit
cd impala-3.4
git fetch origin 481ea4ab0d476a4aa491f99c2a4e376faddc0b03
git cherry-pick 481ea4ab0d476a4aa491f99c2a4e376faddc0b03

# 克隆impala-lzo和hadoop-lzo
git clone --branch master https://github.com/cloudera/impala-lzo.git impala-3.4/../Impala-lzo
git clone https://github.com/cloudera/hadoop-lzo.git impala-3.4/../hadoop-lzo

3. 修改apache-ant下载地址

export IMPALA_HOME=`pwd`
vim $IMPALA_HOME/bin/bootstrap_system.sh

redhat sudo wget -nv \
   https://downloads.apache.org/ant/binaries/apache-ant-1.10.12-bin.tar.gz
redhat sha512sum -c - <<< '2287dc5cfc21043c14e5413f9afb1c87c9f266ec2a9ba2d3bf2285446f6e4ccb59b558bf2e5c57911a05dfa293c7d5c7ad60ac9f744ba11406f4e6f9a27b2403  apache-ant-1.10.12-bin.tar.gz'
redhat sudo tar -C /usr/local -xzf apache-ant-1.10.12-bin.tar.gz
redhat sudo ln -s /usr/local/apache-ant-1.10.12/bin/ant /usr/local/bin

4. 修改ivy下载地址

vim impala-3.4/../hadoop-lzo/build.xml
96   <property name="ivy_repo_url" value="https://repo.maven.apache.org/maven2/org/apache/ivy/ivy/${ivy.version}/ivy-${ivy.version}.jar"/>

vim impala-3.4/../hadoop-lzo/ivy/ivysettings.xml
15    value="https://repo.maven.apache.org/maven2/"

5. 安装编译依赖

$IMPALA_HOME/bin/bootstrap_system.sh

6. 编译

# 设置环境变量
source $IMPALA_HOME/bin/impala-config.sh

# 编译过程会下载python依赖和其他静态依赖，会比较耗费时间
$IMPALA_HOME/buildall.sh -noclean -notests -release

编译完后可以找到impalad可执行文件和impala-frontend的jar包

$ ll -h be/build/latest/service/impalad fe/target/impala-frontend-0.1-SNAPSHOT.jar
-rwxrwxr-x 1 root root 460M 6月  20 11:30 be/build/latest/service/impalad
-rw-rw-r-- 1 root root 7.5M 6月  20 11:33 fe/target/impala-frontend-0.1-SNAPSHOT.jar
$ strings be/build/latest/service/impalad | grep 3.4.0
3.4.0-RELEASE

二、独立安装impala

1. 创建用户

useradd -r -d /var/lib/impala -s /bin/sh -c "impala" impala

2. 构建目录

将编译后的二进制文件、依赖包拷贝出来

mkdir -p /data/apache-impala-3.4/{bin,etc,lib,sbin,toolchain}
mkdir /data/apache-impala-3.4/etc/{init.d,default}

cp $IMPALA_HOME/fe/target/dependency/* /data/apache-impala-3.4/lib/
cp $IMPALA_HOME/fe/target/impala-frontend-0.1-SNAPSHOT.jar /data/apache-impala-3.4/lib/

cp -r $IMPALA_HOME/www /data/apache-impala-3.4/
cp -r $IMPALA_HOME/shell/build/impala-shell-3.4-SNAPSHOT /data/apache-impala-3.4/impala-shell

cp -r $IMPALA_HOME/toolchain/gcc-4.9.2 /data/apache-impala-3.4/toolchain/
cp -r $IMPALA_HOME/toolchain/kudu-4ed0dbbd1 /data/apache-impala-3.4/toolchain/

cp $IMPALA_HOME/be/build/latest/service/libfesupport.so /data/apache-impala-3.4/sbin
cp $IMPALA_HOME/be/build/latest/service/impalad /data/apache-impala-3.4/sbin
ln -s /data/apache-impala-3.4/sbin/impalad /data/apache-impala-3.4/sbin/catalogd
ln -s /data/apache-impala-3.4/sbin/impalad /data/apache-impala-3.4/sbin/statestored

3. 构建启动文件

vim /data/apache-impala-3.4/bin/catalogd

#!/bin/bash

export IMPALA_BIN=${IMPALA_BIN:-/data/apache-impala-3.4/sbin}
export IMPALA_HOME=${IMPALA_HOME:-/data/apache-impala-3.4}
export HIVE_HOME=${HIVE_HOME:-/usr/lib/hive}
export HBASE_HOME=${HBASE_HOME:-/usr/lib/hbase}
export IMPALA_CONF_DIR=${IMPALA_CONF_DIR:-/etc/impala/conf}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/etc/hadoop/conf}
export HIVE_CONF_DIR=${HIVE_CONF_DIR:-/etc/hive/conf}
export HBASE_CONF_DIR=${HBASE_CONF_DIR:-/etc/impala/conf}
export LIBHDFS_OPTS=${LIBHDFS_OPTS:--Djava.library.path=/data/apache-impala-3.4/lib}
export MYSQL_CONNECTOR_JAR=${MYSQL_CONNECTOR_JAR:-/usr/share/java/mysql-connector-java.jar}

if [ "$ENABLE_CORE_DUMPS" == "true" ] ; then
    ulimit -c unlimited
elif [ -z "$ENABLE_CORE_DUMPS" -o "$ENABLE_CORE_DUMPS" == "false" ] ; then
    ulimit -c 0
else
    echo 'WARNING: $ENABLE_CORE_DUMPS must be either "true" or "false"'
fi

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

# ensure that java has already been found
if [ -z "${JAVA_HOME}" ]; then
  echo "Unable to find Java. JAVA_HOME should be set in /etc/default/bigtop-utils"
  exit 1
fi

# Autodetect location of native java libraries
for library in libjvm.so libjsig.so libjava.so; do
    library_file=`find ${JAVA_HOME}/ -name $library | head -1`
    if [ -n "$library_file" ] ; then
        library_dir=`dirname $library_file`
        export LD_LIBRARY_PATH=$library_dir:${LD_LIBRARY_PATH}
    fi
done
export LD_LIBRARY_PATH="${IMPALA_HOME}/lib:${IMPALA_BIN}:${LD_LIBRARY_PATH}"

export CLASSPATH="${IMPALA_HOME}/lib/libthrift-0.9.0.jar::${IMPALA_HOME}/lib/ST4-4.0.4.jar:${IMPALA_HOME}/lib/activation-1.1.jar:${IMPALA_HOME}/lib/ant-1.5.jar:${IMPALA_HOME}/lib/ant-1.9.1.jar:${IMPALA_HOME}/lib/ant-contrib-1.0b3.jar:${IMPALA_HOME}/lib/ant-launcher-1.9.1.jar:${IMPALA_HOME}/lib/antlr-2.7.7.jar:${IMPALA_HOME}/lib/antlr-runtime-3.3.jar:${IMPALA_HOME}/lib/aopalliance-1.0.jar:${IMPALA_HOME}/lib/apache-log4j-extras-1.2.17.jar:${IMPALA_HOME}/lib/apacheds-i18n-2.0.0-M15.jar:${IMPALA_HOME}/lib/apacheds-kerberos-codec-2.0.0-M15.jar:${IMPALA_HOME}/lib/api-asn1-api-1.0.0-M20.jar:${IMPALA_HOME}/lib/api-util-1.0.0-M20.jar:${IMPALA_HOME}/lib/asm-3.1.jar:${IMPALA_HOME}/lib/asm-commons-3.1.jar:${IMPALA_HOME}/lib/asm-tree-3.1.jar:${IMPALA_HOME}/lib/async-1.3.1.jar:${IMPALA_HOME}/lib/avro.jar:${IMPALA_HOME}/lib/aws-java-sdk-core-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-kms-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-s3-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-sts-1.10.6.jar:${IMPALA_HOME}/lib/bonecp-0.7.1.RELEASE.jar:${IMPALA_HOME}/lib/calcite-avatica-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-core-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-linq4j-1.0.0-incubating.jar:${IMPALA_HOME}/lib/commons-beanutils-1.8.3.jar:${IMPALA_HOME}/lib/commons-beanutils-core-1.8.0.jar:${IMPALA_HOME}/lib/commons-cli-1.2.jar:${IMPALA_HOME}/lib/commons-codec-1.9.jar:${IMPALA_HOME}/lib/commons-collections-3.2.2.jar:${IMPALA_HOME}/lib/commons-compiler-2.7.6.jar:${IMPALA_HOME}/lib/commons-compress-1.4.1.jar:${IMPALA_HOME}/lib/commons-configuration-1.6.jar:${IMPALA_HOME}/lib/commons-daemon-1.0.13.jar:${IMPALA_HOME}/lib/commons-dbcp-1.4.jar:${IMPALA_HOME}/lib/commons-digester-1.8.jar:${IMPALA_HOME}/lib/commons-el-1.0.jar:${IMPALA_HOME}/lib/commons-httpclient-3.1.jar:${IMPALA_HOME}/lib/commons-io-2.4.jar:${IMPALA_HOME}/lib/commons-lang-2.6.jar:${IMPALA_HOME}/lib/commons-lang3-3.1.jar:${IMPALA_HOME}/lib/commons-logging-1.1.1.jar:${IMPALA_HOME}/lib/commons-math-2.2.jar:${IMPALA_HOME}/lib/commons-math3-3.1.1.jar:${IMPALA_HOME}/lib/commons-net-3.1.jar:${IMPALA_HOME}/lib/commons-pool-1.5.4.jar:${IMPALA_HOME}/lib/commons-pool2-2.2.jar:${IMPALA_HOME}/lib/core-3.1.1.jar:${IMPALA_HOME}/lib/curator-client-2.7.1.jar:${IMPALA_HOME}/lib/curator-framework-2.7.1.jar:${IMPALA_HOME}/lib/curator-recipes-2.7.1.jar:${IMPALA_HOME}/lib/curator-test-2.7.1.jar:${IMPALA_HOME}/lib/curator-x-discovery-2.7.1.jar:${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.6.jar:${IMPALA_HOME}/lib/datanucleus-core-3.2.12.jar:${IMPALA_HOME}/lib/datanucleus-rdbms-3.2.12.jar:${IMPALA_HOME}/lib/derby-10.10.2.0.jar:${IMPALA_HOME}/lib/eigenbase-properties-1.1.4.jar:${IMPALA_HOME}/lib/findbugs-annotations-1.3.9-1.jar:${IMPALA_HOME}/lib/geronimo-annotation_1.0_spec-1.1.1.jar:${IMPALA_HOME}/lib/geronimo-jaspic_1.0_spec-1.0.jar:${IMPALA_HOME}/lib/geronimo-jta_1.1_spec-1.1.1.jar:${IMPALA_HOME}/lib/groovy-all-2.4.4.jar:${IMPALA_HOME}/lib/gson-2.2.4.jar:${IMPALA_HOME}/lib/guava-11.0.2.jar:${IMPALA_HOME}/lib/guice-3.0.jar:${IMPALA_HOME}/lib/guice-servlet-3.0.jar:${IMPALA_HOME}/lib/hadoop-annotations.jar:${IMPALA_HOME}/lib/hadoop-auth.jar:${IMPALA_HOME}/lib/hadoop-aws.jar:${IMPALA_HOME}/lib/hadoop-common.jar:${IMPALA_HOME}/lib/hadoop-hdfs.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-common.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-core.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-jobclient.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-shuffle.jar:${IMPALA_HOME}/lib/hadoop-yarn-api.jar:${IMPALA_HOME}/lib/hadoop-yarn-client.jar:${IMPALA_HOME}/lib/hadoop-yarn-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-applicationhistoryservice.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-nodemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-resourcemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-web-proxy.jar:${IMPALA_HOME}/lib/hbase-annotations.jar:${IMPALA_HOME}/lib/hbase-client.jar:${IMPALA_HOME}/lib/hbase-common.jar:${IMPALA_HOME}/lib/hbase-protocol.jar:${IMPALA_HOME}/lib/high-scale-lib-1.1.1.jar:${IMPALA_HOME}/lib/hive-ant.jar:${IMPALA_HOME}/lib/hive-beeline.jar:${IMPALA_HOME}/lib/hive-common.jar:${IMPALA_HOME}/lib/hive-exec.jar:${IMPALA_HOME}/lib/hive-hbase-handler.jar:${IMPALA_HOME}/lib/hive-metastore.jar:${IMPALA_HOME}/lib/hive-serde.jar:${IMPALA_HOME}/lib/hive-service.jar:${IMPALA_HOME}/lib/hive-shims-common.jar:${IMPALA_HOME}/lib/hive-shims-scheduler.jar:${IMPALA_HOME}/lib/hive-shims.jar:${IMPALA_HOME}/lib/hsqldb-1.8.0.10.jar:${IMPALA_HOME}/lib/htrace-core-3.0.4.jar:${IMPALA_HOME}/lib/htrace-core-3.2.0-incubating.jar:${IMPALA_HOME}/lib/htrace-core4-4.0.1-incubating.jar:${IMPALA_HOME}/lib/httpclient-4.2.5.jar:${IMPALA_HOME}/lib/httpcore-4.2.5.jar:${IMPALA_HOME}/lib/impala-data-source-api-1.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/impala-frontend-0.1-SNAPSHOT.jar:${IMPALA_HOME}/lib/interface-annotations-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/jackson-annotations-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-databind-2.2.3.jar:${IMPALA_HOME}/lib/jackson-jaxrs-1.8.3.jar:${IMPALA_HOME}/lib/jackson-mapper-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-xc-1.8.3.jar:${IMPALA_HOME}/lib/jamon-runtime-2.3.1.jar:${IMPALA_HOME}/lib/janino-2.7.6.jar:${IMPALA_HOME}/lib/jasper-compiler-5.5.23.jar:${IMPALA_HOME}/lib/jasper-runtime-5.5.23.jar:${IMPALA_HOME}/lib/java-cup-0.11-a-czt02-cdh.jar:${IMPALA_HOME}/lib/java-cup-runtime-0.11-a-czt01-cdh.jar:${IMPALA_HOME}/lib/java-xmlbuilder-0.4.jar:${IMPALA_HOME}/lib/javassist-3.18.1-GA.jar:${IMPALA_HOME}/lib/javax.inject-1.jar:${IMPALA_HOME}/lib/javax.json-1.0.2.jar:${IMPALA_HOME}/lib/javax.servlet-2.5.0.v201103041518.jar:${IMPALA_HOME}/lib/jaxb-api-2.2.2.jar:${IMPALA_HOME}/lib/jaxb-impl-2.2.3-1.jar:${IMPALA_HOME}/lib/jcodings-1.0.8.jar:${IMPALA_HOME}/lib/jdo-api-3.0.1.jar:${IMPALA_HOME}/lib/jersey-client-1.9.jar:${IMPALA_HOME}/lib/jersey-core-1.9.jar:${IMPALA_HOME}/lib/jersey-guice-1.9.jar:${IMPALA_HOME}/lib/jersey-json-1.9.jar:${IMPALA_HOME}/lib/jersey-server-1.9.jar:${IMPALA_HOME}/lib/jersey-servlet-1.14.jar:${IMPALA_HOME}/lib/jets3t-0.9.0.jar:${IMPALA_HOME}/lib/jettison-1.1.jar:${IMPALA_HOME}/lib/jetty-6.1.26.cloudera.4.jar:${IMPALA_HOME}/lib/jetty-all-7.6.0.v20120127.jar:${IMPALA_HOME}/lib/jetty-continuation-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-http-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-io-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-security-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-server-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-servlet-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-util-6.1.26.jar:${IMPALA_HOME}/lib/jetty-util-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jline-2.12.jar:${IMPALA_HOME}/lib/joda-time-2.8.1.jar:${IMPALA_HOME}/lib/joni-2.1.2.jar:${IMPALA_HOME}/lib/jpam-1.1.jar:${IMPALA_HOME}/lib/jsch-0.1.42.jar:${IMPALA_HOME}/lib/json-simple-1.1.1.jar:${IMPALA_HOME}/lib/jsp-api-2.1.jar:${IMPALA_HOME}/lib/jsr305-3.0.0.jar:${IMPALA_HOME}/lib/jta-1.1.jar:${IMPALA_HOME}/lib/kudu-client-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/leveldbjni-all-1.8.jar:${IMPALA_HOME}/lib/libfb303-0.9.0.jar:${IMPALA_HOME}/lib/libgcc_s.so.1:${IMPALA_HOME}/lib/libhadoop.so:${IMPALA_HOME}/lib/libhadoop.so.1.0.0:${IMPALA_HOME}/lib/libhdfs.so:${IMPALA_HOME}/lib/libhdfs.so.0.0.0:${IMPALA_HOME}/lib/libkudu_client.so.0:${IMPALA_HOME}/lib/libkudu_client.so.0.1.0:${IMPALA_HOME}/lib/libstdc++.so.6:${IMPALA_HOME}/lib/libstdc++.so.6.0.20:${IMPALA_HOME}/lib/libthrift-0.9.0.jar:${IMPALA_HOME}/lib/log4j-1.2.17.jar:${IMPALA_HOME}/lib/logredactor-1.0.3.jar:${IMPALA_HOME}/lib/mail-1.4.1.jar:${IMPALA_HOME}/lib/metrics-core-2.2.0.jar:${IMPALA_HOME}/lib/metrics-core-3.0.2.jar:${IMPALA_HOME}/lib/metrics-healthchecks-3.0.2.jar:${IMPALA_HOME}/lib/metrics-json-3.0.2.jar:${IMPALA_HOME}/lib/metrics-jvm-3.0.2.jar:${IMPALA_HOME}/lib/metrics-servlets-3.0.2.jar:${IMPALA_HOME}/lib/mockito-all-1.9.5.jar:${IMPALA_HOME}/lib/netty-3.10.5.Final.jar:${IMPALA_HOME}/lib/netty-all-4.0.23.Final.jar:${IMPALA_HOME}/lib/opencsv-2.3.jar:${IMPALA_HOME}/lib/oro-2.0.8.jar:${IMPALA_HOME}/lib/paranamer-2.3.jar:${IMPALA_HOME}/lib/parquet-hadoop-bundle.jar:${IMPALA_HOME}/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:${IMPALA_HOME}/lib/postgresql-9.0-801.jdbc4.jar:${IMPALA_HOME}/lib/protobuf-java-2.5.0.jar:${IMPALA_HOME}/lib/sentry-binding-hive.jar:${IMPALA_HOME}/lib/sentry-core-common.jar:${IMPALA_HOME}/lib/sentry-core-model-db.jar:${IMPALA_HOME}/lib/sentry-core-model-kafka.jar:${IMPALA_HOME}/lib/sentry-core-model-search.jar:${IMPALA_HOME}/lib/sentry-policy-common.jar:${IMPALA_HOME}/lib/sentry-policy-db.jar:${IMPALA_HOME}/lib/sentry-policy-kafka.jar:${IMPALA_HOME}/lib/sentry-policy-search.jar:${IMPALA_HOME}/lib/sentry-provider-cache.jar:${IMPALA_HOME}/lib/sentry-provider-common.jar:${IMPALA_HOME}/lib/sentry-provider-db-sh.jar:${IMPALA_HOME}/lib/sentry-provider-file.jar:${IMPALA_HOME}/lib/servlet-api-2.5.jar:${IMPALA_HOME}/lib/shiro-core-1.2.3.jar:${IMPALA_HOME}/lib/slf4j-api-1.7.5.jar:${IMPALA_HOME}/lib/slf4j-log4j12-1.7.5.jar:${IMPALA_HOME}/lib/snappy-java-1.0.4.1.jar:${IMPALA_HOME}/lib/stax-api-1.0-2.jar:${IMPALA_HOME}/lib/stax-api-1.0.1.jar:${IMPALA_HOME}/lib/stringtemplate-3.2.1.jar:${IMPALA_HOME}/lib/super-csv-2.2.0.jar:${IMPALA_HOME}/lib/velocity-1.5.jar:${IMPALA_HOME}/lib/xercesImpl-2.9.1.jar:${IMPALA_HOME}/lib/xml-apis-1.3.04.jar:${IMPALA_HOME}/lib/xmlenc-0.52.jar:${IMPALA_HOME}/lib/xz-1.0.jar:${IMPALA_HOME}/lib/zookeeper.jar::${CLASSPATH}"
export CLASSPATH="${IMPALA_CONF_DIR}:${HADOOP_CONF_DIR}:${HIVE_CONF_DIR}:${HBASE_CONF_DIR}:${CLASSPATH}"
export CLASSPATH="${MYSQL_CONNECTOR_JAR}:${CLASSPATH}"
for JAR_FILE in /var/lib/impala/*.jar; do
    export CLASSPATH="${JAR_FILE}:${CLASSPATH}"
done
if [ -n "${AUX_CLASSPATH}" ]; then
    export CLASSPATH="${AUX_CLASSPATH}:${CLASSPATH}"
fi

# Add non-standard kinit location to PATH
if [ -d /usr/kerberos/bin ]; then
  export PATH=/usr/kerberos/bin:${PATH}
fi

exec ${IMPALA_BIN}/catalogd "$@"

vim /data/apache-impala-3.4/bin/impalad

#!/bin/bash

export IMPALA_BIN=${IMPALA_BIN:-/data/apache-impala-3.4/sbin}
export IMPALA_HOME=${IMPALA_HOME:-/data/apache-impala-3.4}
export HIVE_HOME=${HIVE_HOME:-/usr/lib/hive}
export HBASE_HOME=${HBASE_HOME:-/usr/lib/hbase}
export IMPALA_CONF_DIR=${IMPALA_CONF_DIR:-/etc/impala/conf}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/etc/hadoop/conf}
export HIVE_CONF_DIR=${HIVE_CONF_DIR:-/etc/hive/conf}
export HBASE_CONF_DIR=${HBASE_CONF_DIR:-/etc/impala/conf}
export LIBHDFS_OPTS=${LIBHDFS_OPTS:--Djava.library.path=/data/apache-impala-3.4/lib}
export MYSQL_CONNECTOR_JAR=${MYSQL_CONNECTOR_JAR:-/usr/share/java/mysql-connector-java.jar}

if [ "$ENABLE_CORE_DUMPS" == "true" ] ; then
    ulimit -c unlimited
elif [ -z "$ENABLE_CORE_DUMPS" -o "$ENABLE_CORE_DUMPS" == "false" ] ; then
    ulimit -c 0
else
    echo 'WARNING: $ENABLE_CORE_DUMPS must be either "true" or "false"'
fi

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

# ensure that java has already been found
if [ -z "${JAVA_HOME}" ]; then
  echo "Unable to find Java. JAVA_HOME should be set in /etc/default/bigtop-utils"
  exit 1
fi

# Autodetect location of native java libraries
for library in libjvm.so libjsig.so libjava.so; do
    library_file=`find ${JAVA_HOME}/ -name $library | head -1`
    if [ -n "$library_file" ] ; then
        library_dir=`dirname $library_file`
        export LD_LIBRARY_PATH=$library_dir:${LD_LIBRARY_PATH}
    fi
done
export LD_LIBRARY_PATH="${IMPALA_HOME}/lib:${IMPALA_BIN}:${LD_LIBRARY_PATH}"

export CLASSPATH="${IMPALA_HOME}/lib/libthrift-0.9.0.jar::${IMPALA_HOME}/lib/ST4-4.0.4.jar:${IMPALA_HOME}/lib/activation-1.1.jar:${IMPALA_HOME}/lib/ant-1.5.jar:${IMPALA_HOME}/lib/ant-1.9.1.jar:${IMPALA_HOME}/lib/ant-contrib-1.0b3.jar:${IMPALA_HOME}/lib/ant-launcher-1.9.1.jar:${IMPALA_HOME}/lib/antlr-2.7.7.jar:${IMPALA_HOME}/lib/antlr-runtime-3.3.jar:${IMPALA_HOME}/lib/aopalliance-1.0.jar:${IMPALA_HOME}/lib/apache-log4j-extras-1.2.17.jar:${IMPALA_HOME}/lib/apacheds-i18n-2.0.0-M15.jar:${IMPALA_HOME}/lib/apacheds-kerberos-codec-2.0.0-M15.jar:${IMPALA_HOME}/lib/api-asn1-api-1.0.0-M20.jar:${IMPALA_HOME}/lib/api-util-1.0.0-M20.jar:${IMPALA_HOME}/lib/asm-3.1.jar:${IMPALA_HOME}/lib/asm-commons-3.1.jar:${IMPALA_HOME}/lib/asm-tree-3.1.jar:${IMPALA_HOME}/lib/async-1.3.1.jar:${IMPALA_HOME}/lib/avro.jar:${IMPALA_HOME}/lib/aws-java-sdk-core-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-kms-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-s3-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-sts-1.10.6.jar:${IMPALA_HOME}/lib/bonecp-0.7.1.RELEASE.jar:${IMPALA_HOME}/lib/calcite-avatica-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-core-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-linq4j-1.0.0-incubating.jar:${IMPALA_HOME}/lib/commons-beanutils-1.8.3.jar:${IMPALA_HOME}/lib/commons-beanutils-core-1.8.0.jar:${IMPALA_HOME}/lib/commons-cli-1.2.jar:${IMPALA_HOME}/lib/commons-codec-1.9.jar:${IMPALA_HOME}/lib/commons-collections-3.2.2.jar:${IMPALA_HOME}/lib/commons-compiler-2.7.6.jar:${IMPALA_HOME}/lib/commons-compress-1.4.1.jar:${IMPALA_HOME}/lib/commons-configuration-1.6.jar:${IMPALA_HOME}/lib/commons-daemon-1.0.13.jar:${IMPALA_HOME}/lib/commons-dbcp-1.4.jar:${IMPALA_HOME}/lib/commons-digester-1.8.jar:${IMPALA_HOME}/lib/commons-el-1.0.jar:${IMPALA_HOME}/lib/commons-httpclient-3.1.jar:${IMPALA_HOME}/lib/commons-io-2.4.jar:${IMPALA_HOME}/lib/commons-lang-2.6.jar:${IMPALA_HOME}/lib/commons-lang3-3.1.jar:${IMPALA_HOME}/lib/commons-logging-1.1.1.jar:${IMPALA_HOME}/lib/commons-math-2.2.jar:${IMPALA_HOME}/lib/commons-math3-3.1.1.jar:${IMPALA_HOME}/lib/commons-net-3.1.jar:${IMPALA_HOME}/lib/commons-pool-1.5.4.jar:${IMPALA_HOME}/lib/commons-pool2-2.2.jar:${IMPALA_HOME}/lib/core-3.1.1.jar:${IMPALA_HOME}/lib/curator-client-2.7.1.jar:${IMPALA_HOME}/lib/curator-framework-2.7.1.jar:${IMPALA_HOME}/lib/curator-recipes-2.7.1.jar:${IMPALA_HOME}/lib/curator-test-2.7.1.jar:${IMPALA_HOME}/lib/curator-x-discovery-2.7.1.jar:${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.6.jar:${IMPALA_HOME}/lib/datanucleus-core-3.2.12.jar:${IMPALA_HOME}/lib/datanucleus-rdbms-3.2.12.jar:${IMPALA_HOME}/lib/derby-10.10.2.0.jar:${IMPALA_HOME}/lib/eigenbase-properties-1.1.4.jar:${IMPALA_HOME}/lib/findbugs-annotations-1.3.9-1.jar:${IMPALA_HOME}/lib/geronimo-annotation_1.0_spec-1.1.1.jar:${IMPALA_HOME}/lib/geronimo-jaspic_1.0_spec-1.0.jar:${IMPALA_HOME}/lib/geronimo-jta_1.1_spec-1.1.1.jar:${IMPALA_HOME}/lib/groovy-all-2.4.4.jar:${IMPALA_HOME}/lib/gson-2.2.4.jar:${IMPALA_HOME}/lib/guava-11.0.2.jar:${IMPALA_HOME}/lib/guice-3.0.jar:${IMPALA_HOME}/lib/guice-servlet-3.0.jar:${IMPALA_HOME}/lib/hadoop-annotations.jar:${IMPALA_HOME}/lib/hadoop-auth.jar:${IMPALA_HOME}/lib/hadoop-aws.jar:${IMPALA_HOME}/lib/hadoop-common.jar:${IMPALA_HOME}/lib/hadoop-hdfs.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-common.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-core.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-jobclient.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-shuffle.jar:${IMPALA_HOME}/lib/hadoop-yarn-api.jar:${IMPALA_HOME}/lib/hadoop-yarn-client.jar:${IMPALA_HOME}/lib/hadoop-yarn-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-applicationhistoryservice.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-nodemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-resourcemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-web-proxy.jar:${IMPALA_HOME}/lib/hbase-annotations.jar:${IMPALA_HOME}/lib/hbase-client.jar:${IMPALA_HOME}/lib/hbase-common.jar:${IMPALA_HOME}/lib/hbase-protocol.jar:${IMPALA_HOME}/lib/high-scale-lib-1.1.1.jar:${IMPALA_HOME}/lib/hive-ant.jar:${IMPALA_HOME}/lib/hive-beeline.jar:${IMPALA_HOME}/lib/hive-common.jar:${IMPALA_HOME}/lib/hive-exec.jar:${IMPALA_HOME}/lib/hive-hbase-handler.jar:${IMPALA_HOME}/lib/hive-metastore.jar:${IMPALA_HOME}/lib/hive-serde.jar:${IMPALA_HOME}/lib/hive-service.jar:${IMPALA_HOME}/lib/hive-shims-common.jar:${IMPALA_HOME}/lib/hive-shims-scheduler.jar:${IMPALA_HOME}/lib/hive-shims.jar:${IMPALA_HOME}/lib/hsqldb-1.8.0.10.jar:${IMPALA_HOME}/lib/htrace-core-3.0.4.jar:${IMPALA_HOME}/lib/htrace-core-3.2.0-incubating.jar:${IMPALA_HOME}/lib/htrace-core4-4.0.1-incubating.jar:${IMPALA_HOME}/lib/httpclient-4.2.5.jar:${IMPALA_HOME}/lib/httpcore-4.2.5.jar:${IMPALA_HOME}/lib/impala-data-source-api-1.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/impala-frontend-0.1-SNAPSHOT.jar:${IMPALA_HOME}/lib/interface-annotations-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/jackson-annotations-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-databind-2.2.3.jar:${IMPALA_HOME}/lib/jackson-jaxrs-1.8.3.jar:${IMPALA_HOME}/lib/jackson-mapper-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-xc-1.8.3.jar:${IMPALA_HOME}/lib/jamon-runtime-2.3.1.jar:${IMPALA_HOME}/lib/janino-2.7.6.jar:${IMPALA_HOME}/lib/jasper-compiler-5.5.23.jar:${IMPALA_HOME}/lib/jasper-runtime-5.5.23.jar:${IMPALA_HOME}/lib/java-cup-0.11-a-czt02-cdh.jar:${IMPALA_HOME}/lib/java-cup-runtime-0.11-a-czt01-cdh.jar:${IMPALA_HOME}/lib/java-xmlbuilder-0.4.jar:${IMPALA_HOME}/lib/javassist-3.18.1-GA.jar:${IMPALA_HOME}/lib/javax.inject-1.jar:${IMPALA_HOME}/lib/javax.json-1.0.2.jar:${IMPALA_HOME}/lib/javax.servlet-2.5.0.v201103041518.jar:${IMPALA_HOME}/lib/jaxb-api-2.2.2.jar:${IMPALA_HOME}/lib/jaxb-impl-2.2.3-1.jar:${IMPALA_HOME}/lib/jcodings-1.0.8.jar:${IMPALA_HOME}/lib/jdo-api-3.0.1.jar:${IMPALA_HOME}/lib/jersey-client-1.9.jar:${IMPALA_HOME}/lib/jersey-core-1.9.jar:${IMPALA_HOME}/lib/jersey-guice-1.9.jar:${IMPALA_HOME}/lib/jersey-json-1.9.jar:${IMPALA_HOME}/lib/jersey-server-1.9.jar:${IMPALA_HOME}/lib/jersey-servlet-1.14.jar:${IMPALA_HOME}/lib/jets3t-0.9.0.jar:${IMPALA_HOME}/lib/jettison-1.1.jar:${IMPALA_HOME}/lib/jetty-6.1.26.cloudera.4.jar:${IMPALA_HOME}/lib/jetty-all-7.6.0.v20120127.jar:${IMPALA_HOME}/lib/jetty-continuation-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-http-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-io-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-security-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-server-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-servlet-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-util-6.1.26.jar:${IMPALA_HOME}/lib/jetty-util-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jline-2.12.jar:${IMPALA_HOME}/lib/joda-time-2.8.1.jar:${IMPALA_HOME}/lib/joni-2.1.2.jar:${IMPALA_HOME}/lib/jpam-1.1.jar:${IMPALA_HOME}/lib/jsch-0.1.42.jar:${IMPALA_HOME}/lib/json-simple-1.1.1.jar:${IMPALA_HOME}/lib/jsp-api-2.1.jar:${IMPALA_HOME}/lib/jsr305-3.0.0.jar:${IMPALA_HOME}/lib/jta-1.1.jar:${IMPALA_HOME}/lib/kudu-client-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/leveldbjni-all-1.8.jar:${IMPALA_HOME}/lib/libfb303-0.9.0.jar:${IMPALA_HOME}/lib/libgcc_s.so.1:${IMPALA_HOME}/lib/libhadoop.so:${IMPALA_HOME}/lib/libhadoop.so.1.0.0:${IMPALA_HOME}/lib/libhdfs.so:${IMPALA_HOME}/lib/libhdfs.so.0.0.0:${IMPALA_HOME}/lib/libkudu_client.so.0:${IMPALA_HOME}/lib/libkudu_client.so.0.1.0:${IMPALA_HOME}/lib/libstdc++.so.6:${IMPALA_HOME}/lib/libstdc++.so.6.0.20:${IMPALA_HOME}/lib/libthrift-0.9.0.jar:${IMPALA_HOME}/lib/log4j-1.2.17.jar:${IMPALA_HOME}/lib/logredactor-1.0.3.jar:${IMPALA_HOME}/lib/mail-1.4.1.jar:${IMPALA_HOME}/lib/metrics-core-2.2.0.jar:${IMPALA_HOME}/lib/metrics-core-3.0.2.jar:${IMPALA_HOME}/lib/metrics-healthchecks-3.0.2.jar:${IMPALA_HOME}/lib/metrics-json-3.0.2.jar:${IMPALA_HOME}/lib/metrics-jvm-3.0.2.jar:${IMPALA_HOME}/lib/metrics-servlets-3.0.2.jar:${IMPALA_HOME}/lib/mockito-all-1.9.5.jar:${IMPALA_HOME}/lib/netty-3.10.5.Final.jar:${IMPALA_HOME}/lib/netty-all-4.0.23.Final.jar:${IMPALA_HOME}/lib/opencsv-2.3.jar:${IMPALA_HOME}/lib/oro-2.0.8.jar:${IMPALA_HOME}/lib/paranamer-2.3.jar:${IMPALA_HOME}/lib/parquet-hadoop-bundle.jar:${IMPALA_HOME}/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:${IMPALA_HOME}/lib/postgresql-9.0-801.jdbc4.jar:${IMPALA_HOME}/lib/protobuf-java-2.5.0.jar:${IMPALA_HOME}/lib/sentry-binding-hive.jar:${IMPALA_HOME}/lib/sentry-core-common.jar:${IMPALA_HOME}/lib/sentry-core-model-db.jar:${IMPALA_HOME}/lib/sentry-core-model-kafka.jar:${IMPALA_HOME}/lib/sentry-core-model-search.jar:${IMPALA_HOME}/lib/sentry-policy-common.jar:${IMPALA_HOME}/lib/sentry-policy-db.jar:${IMPALA_HOME}/lib/sentry-policy-kafka.jar:${IMPALA_HOME}/lib/sentry-policy-search.jar:${IMPALA_HOME}/lib/sentry-provider-cache.jar:${IMPALA_HOME}/lib/sentry-provider-common.jar:${IMPALA_HOME}/lib/sentry-provider-db-sh.jar:${IMPALA_HOME}/lib/sentry-provider-file.jar:${IMPALA_HOME}/lib/servlet-api-2.5.jar:${IMPALA_HOME}/lib/shiro-core-1.2.3.jar:${IMPALA_HOME}/lib/slf4j-api-1.7.5.jar:${IMPALA_HOME}/lib/slf4j-log4j12-1.7.5.jar:${IMPALA_HOME}/lib/snappy-java-1.0.4.1.jar:${IMPALA_HOME}/lib/stax-api-1.0-2.jar:${IMPALA_HOME}/lib/stax-api-1.0.1.jar:${IMPALA_HOME}/lib/stringtemplate-3.2.1.jar:${IMPALA_HOME}/lib/super-csv-2.2.0.jar:${IMPALA_HOME}/lib/velocity-1.5.jar:${IMPALA_HOME}/lib/xercesImpl-2.9.1.jar:${IMPALA_HOME}/lib/xml-apis-1.3.04.jar:${IMPALA_HOME}/lib/xmlenc-0.52.jar:${IMPALA_HOME}/lib/xz-1.0.jar:${IMPALA_HOME}/lib/zookeeper.jar::${CLASSPATH}"
export CLASSPATH="${IMPALA_CONF_DIR}:${HADOOP_CONF_DIR}:${HIVE_CONF_DIR}:${HBASE_CONF_DIR}:${CLASSPATH}"
export CLASSPATH="${MYSQL_CONNECTOR_JAR}:${CLASSPATH}"
for JAR_FILE in /var/lib/impala/*.jar; do
    export CLASSPATH="${JAR_FILE}:${CLASSPATH}"
done
if [ -n "${AUX_CLASSPATH}" ]; then
    export CLASSPATH="${AUX_CLASSPATH}:${CLASSPATH}"
fi

# Add non-standard kinit location to PATH
if [ -d /usr/kerberos/bin ]; then
  export PATH=/usr/kerberos/bin:${PATH}
fi

exec ${IMPALA_BIN}/impalad "$@"

vim /data/apache-impala-3.4/bin/statestored

#!/bin/bash

export IMPALA_BIN=${IMPALA_BIN:-/data/apache-impala-3.4/sbin}
export IMPALA_HOME=${IMPALA_HOME:-/data/apache-impala-3.4}
export HIVE_HOME=${HIVE_HOME:-/usr/lib/hive}
export HBASE_HOME=${HBASE_HOME:-/usr/lib/hbase}
export IMPALA_CONF_DIR=${IMPALA_CONF_DIR:-/etc/impala/conf}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-/etc/hadoop/conf}
export HIVE_CONF_DIR=${HIVE_CONF_DIR:-/etc/hive/conf}
export HBASE_CONF_DIR=${HBASE_CONF_DIR:-/etc/impala/conf}
export LIBHDFS_OPTS=${LIBHDFS_OPTS:--Djava.library.path=/data/apache-impala-3.4/lib}
export MYSQL_CONNECTOR_JAR=${MYSQL_CONNECTOR_JAR:-/usr/share/java/mysql-connector-java.jar}

if [ "$ENABLE_CORE_DUMPS" == "true" ] ; then
    ulimit -c unlimited
elif [ -z "$ENABLE_CORE_DUMPS" -o "$ENABLE_CORE_DUMPS" == "false" ] ; then
    ulimit -c 0
else
    echo 'WARNING: $ENABLE_CORE_DUMPS must be either "true" or "false"'
fi

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

# ensure that java has already been found
if [ -z "${JAVA_HOME}" ]; then
  echo "Unable to find Java. JAVA_HOME should be set in /etc/default/bigtop-utils"
  exit 1
fi

# Autodetect location of native java libraries
for library in libjvm.so libjsig.so libjava.so; do
    library_file=`find ${JAVA_HOME}/ -name $library | head -1`
    if [ -n "$library_file" ] ; then
        library_dir=`dirname $library_file`
        export LD_LIBRARY_PATH=$library_dir:${LD_LIBRARY_PATH}
    fi
done
export LD_LIBRARY_PATH="${IMPALA_HOME}/lib:${IMPALA_BIN}:${LD_LIBRARY_PATH}"

export CLASSPATH="${IMPALA_HOME}/lib/libthrift-0.9.0.jar::${IMPALA_HOME}/lib/ST4-4.0.4.jar:${IMPALA_HOME}/lib/activation-1.1.jar:${IMPALA_HOME}/lib/ant-1.5.jar:${IMPALA_HOME}/lib/ant-1.9.1.jar:${IMPALA_HOME}/lib/ant-contrib-1.0b3.jar:${IMPALA_HOME}/lib/ant-launcher-1.9.1.jar:${IMPALA_HOME}/lib/antlr-2.7.7.jar:${IMPALA_HOME}/lib/antlr-runtime-3.3.jar:${IMPALA_HOME}/lib/aopalliance-1.0.jar:${IMPALA_HOME}/lib/apache-log4j-extras-1.2.17.jar:${IMPALA_HOME}/lib/apacheds-i18n-2.0.0-M15.jar:${IMPALA_HOME}/lib/apacheds-kerberos-codec-2.0.0-M15.jar:${IMPALA_HOME}/lib/api-asn1-api-1.0.0-M20.jar:${IMPALA_HOME}/lib/api-util-1.0.0-M20.jar:${IMPALA_HOME}/lib/asm-3.1.jar:${IMPALA_HOME}/lib/asm-commons-3.1.jar:${IMPALA_HOME}/lib/asm-tree-3.1.jar:${IMPALA_HOME}/lib/async-1.3.1.jar:${IMPALA_HOME}/lib/avro.jar:${IMPALA_HOME}/lib/aws-java-sdk-core-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-kms-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-s3-1.10.6.jar:${IMPALA_HOME}/lib/aws-java-sdk-sts-1.10.6.jar:${IMPALA_HOME}/lib/bonecp-0.7.1.RELEASE.jar:${IMPALA_HOME}/lib/calcite-avatica-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-core-1.0.0-incubating.jar:${IMPALA_HOME}/lib/calcite-linq4j-1.0.0-incubating.jar:${IMPALA_HOME}/lib/commons-beanutils-1.8.3.jar:${IMPALA_HOME}/lib/commons-beanutils-core-1.8.0.jar:${IMPALA_HOME}/lib/commons-cli-1.2.jar:${IMPALA_HOME}/lib/commons-codec-1.9.jar:${IMPALA_HOME}/lib/commons-collections-3.2.2.jar:${IMPALA_HOME}/lib/commons-compiler-2.7.6.jar:${IMPALA_HOME}/lib/commons-compress-1.4.1.jar:${IMPALA_HOME}/lib/commons-configuration-1.6.jar:${IMPALA_HOME}/lib/commons-daemon-1.0.13.jar:${IMPALA_HOME}/lib/commons-dbcp-1.4.jar:${IMPALA_HOME}/lib/commons-digester-1.8.jar:${IMPALA_HOME}/lib/commons-el-1.0.jar:${IMPALA_HOME}/lib/commons-httpclient-3.1.jar:${IMPALA_HOME}/lib/commons-io-2.4.jar:${IMPALA_HOME}/lib/commons-lang-2.6.jar:${IMPALA_HOME}/lib/commons-lang3-3.1.jar:${IMPALA_HOME}/lib/commons-logging-1.1.1.jar:${IMPALA_HOME}/lib/commons-math-2.2.jar:${IMPALA_HOME}/lib/commons-math3-3.1.1.jar:${IMPALA_HOME}/lib/commons-net-3.1.jar:${IMPALA_HOME}/lib/commons-pool-1.5.4.jar:${IMPALA_HOME}/lib/commons-pool2-2.2.jar:${IMPALA_HOME}/lib/core-3.1.1.jar:${IMPALA_HOME}/lib/curator-client-2.7.1.jar:${IMPALA_HOME}/lib/curator-framework-2.7.1.jar:${IMPALA_HOME}/lib/curator-recipes-2.7.1.jar:${IMPALA_HOME}/lib/curator-test-2.7.1.jar:${IMPALA_HOME}/lib/curator-x-discovery-2.7.1.jar:${IMPALA_HOME}/lib/datanucleus-api-jdo-3.2.6.jar:${IMPALA_HOME}/lib/datanucleus-core-3.2.12.jar:${IMPALA_HOME}/lib/datanucleus-rdbms-3.2.12.jar:${IMPALA_HOME}/lib/derby-10.10.2.0.jar:${IMPALA_HOME}/lib/eigenbase-properties-1.1.4.jar:${IMPALA_HOME}/lib/findbugs-annotations-1.3.9-1.jar:${IMPALA_HOME}/lib/geronimo-annotation_1.0_spec-1.1.1.jar:${IMPALA_HOME}/lib/geronimo-jaspic_1.0_spec-1.0.jar:${IMPALA_HOME}/lib/geronimo-jta_1.1_spec-1.1.1.jar:${IMPALA_HOME}/lib/groovy-all-2.4.4.jar:${IMPALA_HOME}/lib/gson-2.2.4.jar:${IMPALA_HOME}/lib/guava-11.0.2.jar:${IMPALA_HOME}/lib/guice-3.0.jar:${IMPALA_HOME}/lib/guice-servlet-3.0.jar:${IMPALA_HOME}/lib/hadoop-annotations.jar:${IMPALA_HOME}/lib/hadoop-auth.jar:${IMPALA_HOME}/lib/hadoop-aws.jar:${IMPALA_HOME}/lib/hadoop-common.jar:${IMPALA_HOME}/lib/hadoop-hdfs.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-common.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-core.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-jobclient.jar:${IMPALA_HOME}/lib/hadoop-mapreduce-client-shuffle.jar:${IMPALA_HOME}/lib/hadoop-yarn-api.jar:${IMPALA_HOME}/lib/hadoop-yarn-client.jar:${IMPALA_HOME}/lib/hadoop-yarn-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-applicationhistoryservice.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-common.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-nodemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-resourcemanager.jar:${IMPALA_HOME}/lib/hadoop-yarn-server-web-proxy.jar:${IMPALA_HOME}/lib/hbase-annotations.jar:${IMPALA_HOME}/lib/hbase-client.jar:${IMPALA_HOME}/lib/hbase-common.jar:${IMPALA_HOME}/lib/hbase-protocol.jar:${IMPALA_HOME}/lib/high-scale-lib-1.1.1.jar:${IMPALA_HOME}/lib/hive-ant.jar:${IMPALA_HOME}/lib/hive-beeline.jar:${IMPALA_HOME}/lib/hive-common.jar:${IMPALA_HOME}/lib/hive-exec.jar:${IMPALA_HOME}/lib/hive-hbase-handler.jar:${IMPALA_HOME}/lib/hive-metastore.jar:${IMPALA_HOME}/lib/hive-serde.jar:${IMPALA_HOME}/lib/hive-service.jar:${IMPALA_HOME}/lib/hive-shims-common.jar:${IMPALA_HOME}/lib/hive-shims-scheduler.jar:${IMPALA_HOME}/lib/hive-shims.jar:${IMPALA_HOME}/lib/hsqldb-1.8.0.10.jar:${IMPALA_HOME}/lib/htrace-core-3.0.4.jar:${IMPALA_HOME}/lib/htrace-core-3.2.0-incubating.jar:${IMPALA_HOME}/lib/htrace-core4-4.0.1-incubating.jar:${IMPALA_HOME}/lib/httpclient-4.2.5.jar:${IMPALA_HOME}/lib/httpcore-4.2.5.jar:${IMPALA_HOME}/lib/impala-data-source-api-1.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/impala-frontend-0.1-SNAPSHOT.jar:${IMPALA_HOME}/lib/interface-annotations-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/jackson-annotations-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-2.2.3.jar:${IMPALA_HOME}/lib/jackson-core-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-databind-2.2.3.jar:${IMPALA_HOME}/lib/jackson-jaxrs-1.8.3.jar:${IMPALA_HOME}/lib/jackson-mapper-asl-1.9.13.jar:${IMPALA_HOME}/lib/jackson-xc-1.8.3.jar:${IMPALA_HOME}/lib/jamon-runtime-2.3.1.jar:${IMPALA_HOME}/lib/janino-2.7.6.jar:${IMPALA_HOME}/lib/jasper-compiler-5.5.23.jar:${IMPALA_HOME}/lib/jasper-runtime-5.5.23.jar:${IMPALA_HOME}/lib/java-cup-0.11-a-czt02-cdh.jar:${IMPALA_HOME}/lib/java-cup-runtime-0.11-a-czt01-cdh.jar:${IMPALA_HOME}/lib/java-xmlbuilder-0.4.jar:${IMPALA_HOME}/lib/javassist-3.18.1-GA.jar:${IMPALA_HOME}/lib/javax.inject-1.jar:${IMPALA_HOME}/lib/javax.json-1.0.2.jar:${IMPALA_HOME}/lib/javax.servlet-2.5.0.v201103041518.jar:${IMPALA_HOME}/lib/jaxb-api-2.2.2.jar:${IMPALA_HOME}/lib/jaxb-impl-2.2.3-1.jar:${IMPALA_HOME}/lib/jcodings-1.0.8.jar:${IMPALA_HOME}/lib/jdo-api-3.0.1.jar:${IMPALA_HOME}/lib/jersey-client-1.9.jar:${IMPALA_HOME}/lib/jersey-core-1.9.jar:${IMPALA_HOME}/lib/jersey-guice-1.9.jar:${IMPALA_HOME}/lib/jersey-json-1.9.jar:${IMPALA_HOME}/lib/jersey-server-1.9.jar:${IMPALA_HOME}/lib/jersey-servlet-1.14.jar:${IMPALA_HOME}/lib/jets3t-0.9.0.jar:${IMPALA_HOME}/lib/jettison-1.1.jar:${IMPALA_HOME}/lib/jetty-6.1.26.cloudera.4.jar:${IMPALA_HOME}/lib/jetty-all-7.6.0.v20120127.jar:${IMPALA_HOME}/lib/jetty-continuation-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-http-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-io-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-security-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-server-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-servlet-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jetty-util-6.1.26.jar:${IMPALA_HOME}/lib/jetty-util-7.6.16.v20140903.jar:${IMPALA_HOME}/lib/jline-2.12.jar:${IMPALA_HOME}/lib/joda-time-2.8.1.jar:${IMPALA_HOME}/lib/joni-2.1.2.jar:${IMPALA_HOME}/lib/jpam-1.1.jar:${IMPALA_HOME}/lib/jsch-0.1.42.jar:${IMPALA_HOME}/lib/json-simple-1.1.1.jar:${IMPALA_HOME}/lib/jsp-api-2.1.jar:${IMPALA_HOME}/lib/jsr305-3.0.0.jar:${IMPALA_HOME}/lib/jta-1.1.jar:${IMPALA_HOME}/lib/kudu-client-0.10.0-SNAPSHOT.jar:${IMPALA_HOME}/lib/leveldbjni-all-1.8.jar:${IMPALA_HOME}/lib/libfb303-0.9.0.jar:${IMPALA_HOME}/lib/libgcc_s.so.1:${IMPALA_HOME}/lib/libhadoop.so:${IMPALA_HOME}/lib/libhadoop.so.1.0.0:${IMPALA_HOME}/lib/libhdfs.so:${IMPALA_HOME}/lib/libhdfs.so.0.0.0:${IMPALA_HOME}/lib/libkudu_client.so.0:${IMPALA_HOME}/lib/libkudu_client.so.0.1.0:${IMPALA_HOME}/lib/libstdc++.so.6:${IMPALA_HOME}/lib/libstdc++.so.6.0.20:${IMPALA_HOME}/lib/libthrift-0.9.0.jar:${IMPALA_HOME}/lib/log4j-1.2.17.jar:${IMPALA_HOME}/lib/logredactor-1.0.3.jar:${IMPALA_HOME}/lib/mail-1.4.1.jar:${IMPALA_HOME}/lib/metrics-core-2.2.0.jar:${IMPALA_HOME}/lib/metrics-core-3.0.2.jar:${IMPALA_HOME}/lib/metrics-healthchecks-3.0.2.jar:${IMPALA_HOME}/lib/metrics-json-3.0.2.jar:${IMPALA_HOME}/lib/metrics-jvm-3.0.2.jar:${IMPALA_HOME}/lib/metrics-servlets-3.0.2.jar:${IMPALA_HOME}/lib/mockito-all-1.9.5.jar:${IMPALA_HOME}/lib/netty-3.10.5.Final.jar:${IMPALA_HOME}/lib/netty-all-4.0.23.Final.jar:${IMPALA_HOME}/lib/opencsv-2.3.jar:${IMPALA_HOME}/lib/oro-2.0.8.jar:${IMPALA_HOME}/lib/paranamer-2.3.jar:${IMPALA_HOME}/lib/parquet-hadoop-bundle.jar:${IMPALA_HOME}/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:${IMPALA_HOME}/lib/postgresql-9.0-801.jdbc4.jar:${IMPALA_HOME}/lib/protobuf-java-2.5.0.jar:${IMPALA_HOME}/lib/sentry-binding-hive.jar:${IMPALA_HOME}/lib/sentry-core-common.jar:${IMPALA_HOME}/lib/sentry-core-model-db.jar:${IMPALA_HOME}/lib/sentry-core-model-kafka.jar:${IMPALA_HOME}/lib/sentry-core-model-search.jar:${IMPALA_HOME}/lib/sentry-policy-common.jar:${IMPALA_HOME}/lib/sentry-policy-db.jar:${IMPALA_HOME}/lib/sentry-policy-kafka.jar:${IMPALA_HOME}/lib/sentry-policy-search.jar:${IMPALA_HOME}/lib/sentry-provider-cache.jar:${IMPALA_HOME}/lib/sentry-provider-common.jar:${IMPALA_HOME}/lib/sentry-provider-db-sh.jar:${IMPALA_HOME}/lib/sentry-provider-file.jar:${IMPALA_HOME}/lib/servlet-api-2.5.jar:${IMPALA_HOME}/lib/shiro-core-1.2.3.jar:${IMPALA_HOME}/lib/slf4j-api-1.7.5.jar:${IMPALA_HOME}/lib/slf4j-log4j12-1.7.5.jar:${IMPALA_HOME}/lib/snappy-java-1.0.4.1.jar:${IMPALA_HOME}/lib/stax-api-1.0-2.jar:${IMPALA_HOME}/lib/stax-api-1.0.1.jar:${IMPALA_HOME}/lib/stringtemplate-3.2.1.jar:${IMPALA_HOME}/lib/super-csv-2.2.0.jar:${IMPALA_HOME}/lib/velocity-1.5.jar:${IMPALA_HOME}/lib/xercesImpl-2.9.1.jar:${IMPALA_HOME}/lib/xml-apis-1.3.04.jar:${IMPALA_HOME}/lib/xmlenc-0.52.jar:${IMPALA_HOME}/lib/xz-1.0.jar:${IMPALA_HOME}/lib/zookeeper.jar::${CLASSPATH}"
export CLASSPATH="${IMPALA_CONF_DIR}:${HADOOP_CONF_DIR}:${HIVE_CONF_DIR}:${HBASE_CONF_DIR}:${CLASSPATH}"
export CLASSPATH="${MYSQL_CONNECTOR_JAR}:${CLASSPATH}"
for JAR_FILE in /var/lib/impala/*.jar; do
    export CLASSPATH="${JAR_FILE}:${CLASSPATH}"
done
if [ -n "${AUX_CLASSPATH}" ]; then
    export CLASSPATH="${AUX_CLASSPATH}:${CLASSPATH}"
fi

# Add non-standard kinit location to PATH
if [ -d /usr/kerberos/bin ]; then
  export PATH=/usr/kerberos/bin:${PATH}
fi

exec ${IMPALA_BIN}/statestored "$@"

vim /data/apache-impala-3.4/etc/init.d/impala-catalog

#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Starts a Impala Catalog Server
#
# chkconfig: 345 91 9
# description: Impala Catalog Server
#
### BEGIN INIT INFO
# Provides:          catalogd
# Short-Description: Impala Catalog Server
# Default-Start:     3 4 5
# Default-Stop:      0 1 2 6
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Should-Start:
# Should-Stop:
### END INIT INFO

. /lib/lsb/init-functions
BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default}
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hadoop ] && . ${BIGTOP_DEFAULTS_DIR}/hadoop
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/impala ] && . ${BIGTOP_DEFAULTS_DIR}/impala

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

RETVAL_SUCCESS=0

STATUS_RUNNING=0
STATUS_DEAD=1
STATUS_DEAD_AND_LOCK=2
STATUS_NOT_RUNNING=3
STATUS_OTHER_ERROR=102


ERROR_PROGRAM_NOT_INSTALLED=5
ERROR_PROGRAM_NOT_CONFIGURED=6


RETVAL=0
SLEEP_TIME=5
PROC_NAME="java"

DAEMON="catalogd"
DESC="Impala Catalog Server"
EXEC_PATH="/usr/bin/catalogd"
EXEC_DIR=""
SVC_USER="impala"
DAEMON_FLAGS="${IMPALA_CATALOG_ARGS}"
CONF_DIR="/etc/impala/conf"
PIDFILE="/var/run/impala/catalogd-impala.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/catalogd"
WORKING_DIR="~/"

install -d -m 0755 -o impala -g impala /var/run/impala 1>/dev/null 2>&1 || :
[ -d "$LOCKDIR" ] || install -d -m 0755 $LOCKDIR 1>/dev/null 2>&1 || :

export ENABLE_CORE_DUMPS
export IMPALA_HOME
export IMPALA_BIN
export IMPALA_CONF_DIR
export HADOOP_CONF_DIR
export HIVE_CONF_DIR
export HBASE_CONF_DIR
export LIBHDFS_OPTS
export MYSQL_CONNECTOR
export HIVE_HOME
export HBASE_HOME

start() {
  [ -x $EXEC_PATH ] || exit $ERROR_PROGRAM_NOT_INSTALLED
  [ -d $CONF_DIR ] || exit $ERROR_PROGRAM_NOT_CONFIGURED

  checkstatus >/dev/null 2>/dev/null
  status=$?
  if [ "$status" -eq "$STATUS_RUNNING" ]; then
    log_success_msg "${DESC} is running"
    exit 0
  fi

  /bin/su -s /bin/bash -c "/bin/bash -c 'cd ~ && echo \$\$ > ${PIDFILE} && exec ${EXEC_PATH} ${DAEMON_FLAGS} >>${IMPALA_LOG_DIR}/impala-catalog.log 2>&1' &" $SVC_USER
  RETVAL=$?

  if [ $RETVAL -eq $STATUS_RUNNING ]; then
    touch $LOCKFILE
    log_success_msg "Started ${DESC} (${DAEMON}) : "
  else
    log_failure_msg "Failed to start ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
stop() {
  killproc -p $PIDFILE $EXEC_PATH
  RETVAL=$?

  if [ $RETVAL -eq $RETVAL_SUCCESS ]; then
    log_success_msg "Stopped ${DESC}: "
    rm -f $LOCKFILE $PIDFILE
  else
    log_failure_msg "Failure to stop ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
restart() {
  stop
  start
}

checkstatusofproc(){
  pidofproc -p $PIDFILE $PROC_NAME > /dev/null
}

checkstatus(){
  checkstatusofproc
  status=$?

  case "$status" in
    $STATUS_RUNNING)
      log_success_msg "${DESC} is running"
      ;;
    $STATUS_DEAD)
      log_failure_msg "${DESC} is dead and pid file exists"
      ;;
    $STATUS_DEAD_AND_LOCK)
      log_failure_msg "${DESC} is dead and lock file exists"
      ;;
    $STATUS_NOT_RUNNING)
      log_failure_msg "${DESC} is not running"
      ;;
    *)
      log_failure_msg "${DESC} status is unknown"
      ;;
  esac
  return $status
}

condrestart(){
  [ -e $LOCKFILE ] && restart || :
}

check_for_root() {
  if [ $(id -ur) -ne 0 ]; then
    echo 'Error: root user required'
    echo
    exit 1
  fi
}

service() {
  case "$1" in
    start)
      check_for_root
      start
      ;;
    stop)
      check_for_root
      stop
      ;;
    status)
      checkstatus
      RETVAL=$?
      ;;
    restart)
      check_for_root
      restart
      ;;
    condrestart|try-restart)
      check_for_root
      condrestart
      ;;
    *)
      echo $"Usage: $0 {start|stop|status|restart|try-restart|condrestart}"
      exit 1
  esac
}

service "$@"

exit $RETVAL

vim /data/apache-impala-3.4/etc/init.d/impala-server

#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Starts a Impala Server
#
# chkconfig: 345 92 8
# description: Impala Server
#
### BEGIN INIT INFO
# Provides:          impalad
# Short-Description: Impala Server
# Default-Start:     3 4 5
# Default-Stop:      0 1 2 6
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Should-Start:
# Should-Stop:
### END INIT INFO

. /lib/lsb/init-functions
BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default}
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hadoop ] && . ${BIGTOP_DEFAULTS_DIR}/hadoop
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/impala ] && . ${BIGTOP_DEFAULTS_DIR}/impala

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

RETVAL_SUCCESS=0

STATUS_RUNNING=0
STATUS_DEAD=1
STATUS_DEAD_AND_LOCK=2
STATUS_NOT_RUNNING=3
STATUS_OTHER_ERROR=102


ERROR_PROGRAM_NOT_INSTALLED=5
ERROR_PROGRAM_NOT_CONFIGURED=6


RETVAL=0
SLEEP_TIME=5
PROC_NAME="java"

DAEMON="impalad"
DESC="Impala Server"
EXEC_PATH="/usr/bin/impalad"
EXEC_DIR=""
SVC_USER="impala"
DAEMON_FLAGS="${IMPALA_SERVER_ARGS}"
CONF_DIR="/etc/impala/conf"
PIDFILE="/var/run/impala/impalad-impala.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/impalad"
WORKING_DIR="~/"

install -d -m 0755 -o impala -g impala /var/run/impala 1>/dev/null 2>&1 || :
[ -d "$LOCKDIR" ] || install -d -m 0755 $LOCKDIR 1>/dev/null 2>&1 || :

export ENABLE_CORE_DUMPS
export IMPALA_HOME
export IMPALA_BIN
export IMPALA_CONF_DIR
export HADOOP_CONF_DIR
export HIVE_CONF_DIR
export HBASE_CONF_DIR
export LIBHDFS_OPTS
export MYSQL_CONNECTOR
export HIVE_HOME
export HBASE_HOME

start() {
  [ -x $EXEC_PATH ] || exit $ERROR_PROGRAM_NOT_INSTALLED
  [ -d $CONF_DIR ] || exit $ERROR_PROGRAM_NOT_CONFIGURED

  checkstatus >/dev/null 2>/dev/null
  status=$?
  if [ "$status" -eq "$STATUS_RUNNING" ]; then
    log_success_msg "${DESC} is running"
    exit 0
  fi


  /bin/su -s /bin/bash -c "/bin/bash -c 'cd ~ && echo \$\$ > ${PIDFILE} && exec ${EXEC_PATH} ${DAEMON_FLAGS} >>${IMPALA_LOG_DIR}/impala-server.log 2>&1' &" $SVC_USER
  RETVAL=$?

  if [ $RETVAL -eq $STATUS_RUNNING ]; then
    touch $LOCKFILE
    log_success_msg "Started ${DESC} (${DAEMON}): "
  else
    log_failure_msg "Failed to start ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
stop() {
  killproc -p $PIDFILE $EXEC_PATH
  RETVAL=$?

  if [ $RETVAL -eq $RETVAL_SUCCESS ]; then
    log_success_msg "Stopped ${DESC}: "
    rm -f $LOCKFILE $PIDFILE
  else
    log_failure_msg "Failure to stop ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
restart() {
  stop
  start
}

checkstatusofproc(){
  pidofproc -p $PIDFILE $PROC_NAME > /dev/null
}

checkstatus(){
  checkstatusofproc
  status=$?

  case "$status" in
    $STATUS_RUNNING)
      log_success_msg "${DESC} is running"
      ;;
    $STATUS_DEAD)
      log_failure_msg "${DESC} is dead and pid file exists"
      ;;
    $STATUS_DEAD_AND_LOCK)
      log_failure_msg "${DESC} is dead and lock file exists"
      ;;
    $STATUS_NOT_RUNNING)
      log_failure_msg "${DESC} is not running"
      ;;
    *)
      log_failure_msg "${DESC} status is unknown"
      ;;
  esac
  return $status
}

condrestart(){
  [ -e $LOCKFILE ] && restart || :
}

check_for_root() {
  if [ $(id -ur) -ne 0 ]; then
    echo 'Error: root user required'
    echo
    exit 1
  fi
}

service() {
  case "$1" in
    start)
      check_for_root
      start
      ;;
    stop)
      check_for_root
      stop
      ;;
    status)
      checkstatus
      RETVAL=$?
      ;;
    restart)
      check_for_root
      restart
      ;;
    condrestart|try-restart)
      check_for_root
      condrestart
      ;;
    *)
      echo $"Usage: $0 {start|stop|status|restart|try-restart|condrestart}"
      exit 1
  esac
}

service "$@"

exit $RETVAL

vim /data/apache-impala-3.4/etc/init.d/impala-state-store

#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Starts a Impala State Store Server
#
# chkconfig: 345 90 10
# description: Impala State Store Server
#
### BEGIN INIT INFO
# Provides:          statestored
# Short-Description: Impala State Store Server
# Default-Start:     3 4 5
# Default-Stop:      0 1 2 6
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Should-Start:
# Should-Stop:
### END INIT INFO

. /lib/lsb/init-functions
BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default}
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hadoop ] && . ${BIGTOP_DEFAULTS_DIR}/hadoop
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/impala ] && . ${BIGTOP_DEFAULTS_DIR}/impala

# Autodetect JAVA_HOME if not defined
. /usr/lib/bigtop-utils/bigtop-detect-javahome

RETVAL_SUCCESS=0

STATUS_RUNNING=0
STATUS_DEAD=1
STATUS_DEAD_AND_LOCK=2
STATUS_NOT_RUNNING=3
STATUS_OTHER_ERROR=102


ERROR_PROGRAM_NOT_INSTALLED=5
ERROR_PROGRAM_NOT_CONFIGURED=6


RETVAL=0
SLEEP_TIME=5
PROC_NAME="java"

DAEMON="statestored"
DESC="Impala State Store Server"
EXEC_PATH="/usr/bin/statestored"
EXEC_DIR=""
SVC_USER="impala"
DAEMON_FLAGS="${IMPALA_STATE_STORE_ARGS}"
CONF_DIR="/etc/impala/conf"
PIDFILE="/var/run/impala/statestored-impala.pid"
LOCKDIR="/var/lock/subsys"
LOCKFILE="$LOCKDIR/statestored"
WORKING_DIR="~/"

install -d -m 0755 -o impala -g impala /var/run/impala 1>/dev/null 2>&1 || :
[ -d "$LOCKDIR" ] || install -d -m 0755 $LOCKDIR 1>/dev/null 2>&1 || :

export ENABLE_CORE_DUMPS
export IMPALA_HOME
export IMPALA_BIN
export IMPALA_CONF_DIR
export HADOOP_CONF_DIR
export HIVE_CONF_DIR
export HBASE_CONF_DIR
export LIBHDFS_OPTS
export MYSQL_CONNECTOR
export HIVE_HOME
export HBASE_HOME

start() {
  [ -x $EXEC_PATH ] || exit $ERROR_PROGRAM_NOT_INSTALLED
  [ -d $CONF_DIR ] || exit $ERROR_PROGRAM_NOT_CONFIGURED

  checkstatus >/dev/null 2>/dev/null
  status=$?
  if [ "$status" -eq "$STATUS_RUNNING" ]; then
    log_success_msg "${DESC} is running"
    exit 0
  fi

  /bin/su -s /bin/bash -c "/bin/bash -c 'cd ~ && echo \$\$ > ${PIDFILE} && exec ${EXEC_PATH} ${DAEMON_FLAGS} >>${IMPALA_LOG_DIR}/impala-state-store.log 2>&1' &" $SVC_USER
  RETVAL=$?

  if [ $RETVAL -eq $STATUS_RUNNING ]; then
    touch $LOCKFILE
    log_success_msg "Started ${DESC} (${DAEMON}): "
  else
    log_failure_msg "Failed to start ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
stop() {
  killproc -p $PIDFILE $EXEC_PATH
  RETVAL=$?

  if [ $RETVAL -eq $RETVAL_SUCCESS ]; then
    log_success_msg "Stopped ${DESC}: "
    rm -f $LOCKFILE $PIDFILE
  else
    log_failure_msg "Failure to stop ${DESC}. Return value: $RETVAL"
  fi

  return $RETVAL
}
restart() {
  stop
  start
}

checkstatusofproc(){
  pidofproc -p $PIDFILE $PROC_NAME > /dev/null
}

checkstatus(){
  checkstatusofproc
  status=$?

  case "$status" in
    $STATUS_RUNNING)
      log_success_msg "${DESC} is running"
      ;;
    $STATUS_DEAD)
      log_failure_msg "${DESC} is dead and pid file exists"
      ;;
    $STATUS_DEAD_AND_LOCK)
      log_failure_msg "${DESC} is dead and lock file exists"
      ;;
    $STATUS_NOT_RUNNING)
      log_failure_msg "${DESC} is not running"
      ;;
    *)
      log_failure_msg "${DESC} status is unknown"
      ;;
  esac
  return $status
}

condrestart(){
  [ -e $LOCKFILE ] && restart || :
}

check_for_root() {
  if [ $(id -ur) -ne 0 ]; then
    echo 'Error: root user required'
    echo
    exit 1
  fi
}

service() {
  case "$1" in
    start)
      check_for_root
      start
      ;;
    stop)
      check_for_root
      stop
      ;;
    status)
      checkstatus
      RETVAL=$?
      ;;
    restart)
      check_for_root
      restart
      ;;
    condrestart|try-restart)
      check_for_root
      condrestart
      ;;
    *)
      echo $"Usage: $0 {start|stop|status|restart|try-restart|condrestart}"
      exit 1
  esac
}

service "$@"

exit $RETVAL

4. 构建配置文件

vim /data/apache-impala-3.4/etc/default/impala （参数值按实际修改）
注：FI集群启动了kerberos，所以这里配置了kerberos相关配置，如果没有启动kerberos，那可删除-kerberos_reinit_interval、-principal、-keytab_file三个参数

IMPALA_CATALOG_SERVICE_HOST=hd6
IMPALA_STATE_STORE_HOST=hd6
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
IMPALA_WEB_ROOT=/data/apache-impala-3.4
IMPALA_PRINCIPAL=impalad/hd6@HADOOP.COM
IMPALA_KEYTAB=/etc/impala/conf/impalad.keytab

IMPALA_CATALOG_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -webserver_doc_root=${IMPALA_WEB_ROOT} \
    -hostname=hd6 \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -kerberos_reinit_interval=60 \
    -principal=${IMPALA_PRINCIPAL} \
    -keytab_file=${IMPALA_KEYTAB}"
IMPALA_STATE_STORE_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -webserver_doc_root=${IMPALA_WEB_ROOT} \
    -hostname=hd6 \
    -kerberos_reinit_interval=60 \
    -principal=${IMPALA_PRINCIPAL} \
    -keytab_file=${IMPALA_KEYTAB}"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT} \
    -webserver_doc_root=${IMPALA_WEB_ROOT} \
    -hostname=hd6 \
    -kerberos_reinit_interval=60 \
    -principal=${IMPALA_PRINCIPAL} \
    -be_principal=${IMPALA_PRINCIPAL} \
    -keytab_file=${IMPALA_KEYTAB}"

ENABLE_CORE_DUMPS=false

# LIBHDFS_OPTS=-Djava.library.path=/data/apache-impala-3.4/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/data/apache-impala-3.4/sbin
# IMPALA_HOME=/data/apache-impala-3.4
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf
# HBASE_CONF_DIR=/etc/impala/conf

for jar in /data/apache-impala-3.4/lib/*.jar; do
  if [ -e "$jar" ] ; then
    CLASSPATH="${CLASSPATH}:$jar"
  fi
done

export CLASSPATH

KUDU_LIB_DIR=/data/apache-impala-3.4/toolchain/kudu-4ed0dbbd1/release/lib64
GCC_LIB_DIR=/data/apache-impala-3.4/toolchain/gcc-4.9.2/lib64

export LD_LIBRARY_PATH="${KUDU_LIB_DIR}:${GCC_LIB_DIR}:${LD_LIBRARY_PATH}"

vim /data/apache-impala-3.4/etc/default/hadoop (hadoop路径按实际修改)

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
export HADOOP_HOME_WARN_SUPPRESS=true
export HADOOP_PREFIX=/data/hadoop-3.1.1

export HADOOP_LIBEXEC_DIR=/data/hadoop-3.1.1/libexec
export HADOOP_CONF_DIR=/etc/hadoop/conf

export HADOOP_COMMON_HOME=/data/hadoop-3.1.1
export HADOOP_HDFS_HOME=/data/hadoop-3.1.1
export HADOOP_YARN_HOME=/data/hadoop-3.1.1

# Set HADOOP_MAPRED_HOME to /usr/lib/hadoop-0.20-mapreduce to use MR1
export HADOOP_MAPRED_HOME=/data/hadoop-3.1.1

export JSVC_HOME=/usr/lib/bigtop-utils

5. 系统操作

mkdir -p /var/lib/impala
mkdir -p /var/log/impala
mkdir -p /etc/impala/conf

chown impala:impala /var/lib/impala
chown impala:impala /var/log/impala
chown impala:impala -R /data/apache-impala-3.4

chmod 775 /data/apache-impala-3.4/bin/*
chmod 775 /data/apache-impala-3.4/etc/init.d/*

ln -s /data/apache-impala-3.4/bin/catalogd /usr/bin/catalogd
ln -s /data/apache-impala-3.4/bin/impalad /usr/bin/impalad
ln -s /data/apache-impala-3.4/bin/statestored /usr/bin/statestored

ln -s /data/apache-impala-3.4/etc/init.d/impala-catalog /etc/init.d/impala-catalog
ln -s /data/apache-impala-3.4/etc/init.d/impala-server /etc/init.d/impala-server
ln -s /data/apache-impala-3.4/etc/init.d/impala-state-store /etc/init.d/impala-state-store

ln -s /data/apache-impala-3.4/etc/default/impala /etc/default/impala
ln -s /data/apache-impala-3.4/etc/default/hadoop /etc/default/hadoop

6. 依赖包

impala独立安装有两个依赖，一个mysql驱动包，一个bigtop-utils
mysql驱动包从网上下载，放到//usr/share/java/mysql-connector-java.jar即可。
bigtop-utils从CDH6里拷贝一份，放到/usr/lib/bigtop-utils

三、集成FI

1. 拷贝配置文件

1) 将FI集群的hdfs-site.xml、 core-site.xml拷贝到/etc/hadoop/conf
2) 将FI集群的hive-site.xml、hivemetastore-site.xml合并成一个hive-site.xml文件，拷贝到/etc/hive/conf

2. 创建用户

在FI集群管理界面，创建一个impalad的机机用户（impala用户是FI保留用户，不允许创建），并添加到supergroup、hadoop、hive组，这样impalad用户有权限访问数仓路径。

3. 创建keytab

在上一步创建了impalad用户后，会默认创建一个impalad@relam的principal，但是impala组件内部通信是通过impalad/_HOSTS格式，所以需要手动创建对应的principal

#FI的默认密码是Admin@123
kadmin -p kadmin/admin
kadmin:  add_principal -randkey impalad/hd6
kadmin:  xst -k /etc/impala/conf/impalad.keytab impalad/hd6

注意：这里测试环境只有一台impald节点，如果实际有多台impald节点，那么需要创建多个impalad/_HOSTS principal，并将多个principal合并到同一个keytab上。
创建完keytab后，在本文上面构建的/etc/default/impala里查看修改实际的principal、keytab路径。

4. 启动impala

service impala-state-store start
service impala-catalog start
service impala-server start

5. 验证

注意：使用impala-shell的时候，默认的service name是impala，但是因FI保留用户问题，本文使用的是impalad进行通信，所以需要通过-s参数指定service name为impalad，如不指定，无法连接到impala访问

/data/apache-impala-3.4/impala-shell/impala-shell -s impalad -k

进行简单的创建表、插入、查询：