安装 Performance Monitor

2017/7/16 浏览量: - | posted in  Greenplum comments

参考:

Installing and enabling Greenplum Command Center

Installation GreenPlum Performance Monitor

GreenPlum安装性能监控工具GPCCC(Greenplum Command Center Console)

前期准备:

a. 创建greenplum监控用数据库(gpperfmon)

b. 创建greenplum监控用数据库角色(gpmon)

c. 配置greenplum数据库接受来自perfmon监控的链接文件(pg_hba.conf和.pgpass)

d. 设置postgresql.conf文件,增加启用监控的参数。(这些参数默认会添加在文件的末尾)

gp_enable_gpperfmon=on
gpperfmon_port=8888
gp_external_enable_exec=on
gpperfmon_log_alert_level=warning

e. 设置pg_hba.conf文件,增加如下信息:

      local    gpperfmon         gpmon         md5
      host     all               gpmon         127.0.0.1/28    md5

安装依赖

yum install vixie-cron -y # 若 gpcc要安装 workload manaager,需要安装这个

安装gpperfmon

[gpadmin@dbaone ~]$ gpperfmon_install --enable --password 123456 --port 5432

20160412:16:10:01:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql -f /opt/greenplum/greenplum-db/./lib/gpperfmon/gpperfmon3.sql template1 >& /dev/null
20160412:16:10:11:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql -f /opt/greenplum/greenplum-db/./lib/gpperfmon/gpperfmon4.sql gpperfmon >& /dev/null
20160412:16:10:12:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql -f /opt/greenplum/greenplum-db/./lib/gpperfmon/gpperfmon41.sql gpperfmon >& /dev/null
20160412:16:10:14:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql -f /opt/greenplum/greenplum-db/./lib/gpperfmon/gpperfmon42.sql gpperfmon >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql -f /opt/greenplum/greenplum-db/./lib/gpperfmon/gpperfmonC.sql template1 >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql template1 -c "DROP ROLE IF EXISTS gpmon" >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 psql template1 -c "CREATE ROLE gpmon WITH SUPERUSER CREATEDB LOGIN ENCRYPTED PASSWORD '123456'" >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-echo "local gpperfmon gpmon md5" >> /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1/pg_hba.conf
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-echo "host all gpmon 127.0.0.1/28 md5" >> /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1/pg_hba.conf
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-touch /home/gpadmin/.pgpass >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-mv -f /home/gpadmin/.pgpass /home/gpadmin/.pgpass.1460448601 >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-echo "*:5432:gpperfmon:gpmon:123456" >> /home/gpadmin/.pgpass
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-cat /home/gpadmin/.pgpass.1460448601 >> /home/gpadmin/.pgpass
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-chmod 0600 /home/gpadmin/.pgpass >& /dev/null
20160412:16:10:17:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 gpconfig -c gp_enable_gpperfmon -v on >& /dev/null
20160412:16:10:23:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 gpconfig -c gpperfmon_port -v 8888 >& /dev/null
20160412:16:10:29:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 gpconfig -c gp_external_enable_exec -v on --masteronly >& /dev/null
20160412:16:10:35:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-PGPORT=5432 gpconfig -c gpperfmon_log_alert_level -v warning >& /dev/null
20160412:16:10:40:019577 gpperfmon_install:dbaone:gpadmin-[INFO]:-gpperfmon will be enabled after a full restart of GPDB

 # 跑一些脚本,创建表和视图

[gpadmin@dbaone ~]\( ll \)GPHOME/lib/gpperfmon/
total 56
-rwxr-xr-x 1 gpadmin gpadmin 18339 Nov 13 15:44 gpperfmon3.sql
-rwxr-xr-x 1 gpadmin gpadmin 9051 Nov 13 15:44 gpperfmon41.sql
-rwxr-xr-x 1 gpadmin gpadmin 14407 Nov 13 15:44 gpperfmon42.sql
-rwxr-xr-x 1 gpadmin gpadmin 3277 Nov 13 15:44 gpperfmon4.sql
-rwxr-xr-x 1 gpadmin gpadmin 250 Nov 13 15:44 gpperfmonC.sql

 # 重启gp生效

[gpadmin@dbaone ~]$ gpstop -afr

20160412:16:40:14:020911 gpstop:dbaone:gpadmin-[INFO]:-Starting gpstop with args: -afr
20160412:16:40:14:020911 gpstop:dbaone:gpadmin-[INFO]:-Gathering information and validating the environment...
20160412:16:40:14:020911 gpstop:dbaone:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20160412:16:40:14:020911 gpstop:dbaone:gpadmin-[INFO]:-Obtaining Segment details from master...
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1'
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-There are 0 connections to the database
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Commencing Master instance shutdown with mode='fast'
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Master host=dbaone
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Detected 0 connections to database
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Using standard WAIT mode of 120 seconds
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Commencing Master instance shutdown with mode=fast
20160412:16:40:15:020911 gpstop:dbaone:gpadmin-[INFO]:-Master segment instance directory=/opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1
20160412:16:40:16:020911 gpstop:dbaone:gpadmin-[INFO]:-Attempting forceful termination of any leftover master process
20160412:16:40:16:020911 gpstop:dbaone:gpadmin-[INFO]:-Terminating processes for segment /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1
20160412:16:40:16:020911 gpstop:dbaone:gpadmin-[INFO]:-Stopping master standby host dbafour mode=fast
20160412:16:40:17:020911 gpstop:dbaone:gpadmin-[INFO]:-Successfully shutdown standby process on dbafour
20160412:16:40:17:020911 gpstop:dbaone:gpadmin-[INFO]:-Commencing parallel primary segment instance shutdown, please wait...
20160412:16:40:17:020911 gpstop:dbaone:gpadmin-[INFO]:-0.00% of jobs completed
20160412:16:40:27:020911 gpstop:dbaone:gpadmin-[INFO]:-100.00% of jobs completed
20160412:16:40:27:020911 gpstop:dbaone:gpadmin-[INFO]:-Commencing parallel mirror segment instance shutdown, please wait...
20160412:16:40:27:020911 gpstop:dbaone:gpadmin-[INFO]:-0.00% of jobs completed
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-100.00% of jobs completed
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-----------------------------------------------------
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:- Segments stopped successfully = 12
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:- Segments with errors during stop = 0
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-----------------------------------------------------
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-Successfully shutdown 12 of 12 segment instances
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-Database successfully shutdown with no errors reported
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-Cleaning up leftover gpmmon process
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-No leftover gpmmon process found
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-Cleaning up leftover gpsmon processes
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-No leftover gpsmon processes on some hosts. not attempting forceful termination on these hosts
20160412:16:40:37:020911 gpstop:dbaone:gpadmin-[INFO]:-Cleaning up leftover shared memory
20160412:16:40:42:020911 gpstop:dbaone:gpadmin-[INFO]:-Restarting System...

[gpadmin@dbaone ~]\( ps -ef | grep gpmmon | grep -v grep
gpadmin 21113 21104 0 16:40 ? 00:00:00 /opt/greenplum/greenplum-db-4.3.6.2/bin/gpmmon -D /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1/gpperfmon/conf/gpperfmon.conf -p 5432
[gpadmin@dbaone ~]\)
[gpadmin@dbaone ~]$ ps -ef | grep greenplum | grep -v grep
gpadmin 21104 1 0 16:40 ? 00:00:00 /opt/greenplum/greenplum-db-4.3.6.2/bin/postgres -D /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1 -p 5432 -b 1 -z 6 --silent-mode=true -i -M master -C -1 -x 14 -E
gpadmin 21113 21104 0 16:40 ? 00:00:00 /opt/greenplum/greenplum-db-4.3.6.2/bin/gpmmon -D /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1/gpperfmon/conf/gpperfmon.conf -p 5432
gpadmin 21393 1 0 16:40 ? 00:00:00 /opt/greenplum/greenplum-db/./bin/gpsmon -m 0 -i -t 150 -l /opt/greenplum/greenplum-db/gpdata/gpmaster/gpseg-1/gpperfmon/logs -v 0 8888

[gpadmin@dbaone ~]$ psql
psql (8.2.15)
Type "help" for help.

testdb=# \l
List of databases
Name | Owner | Encoding | Access privileges
-----------+---------+----------+---------------------
gpperfmon | gpadmin | UTF8 | gpadmin=CTc/gpadmin
: =c/gpadmin
postgres | gpadmin | UTF8 |
template0 | gpadmin | UTF8 | =c/gpadmin
: gpadmin=CTc/gpadmin
template1 | gpadmin | UTF8 | =c/gpadmin
: gpadmin=CTc/gpadmin
testdb | gpadmin | UTF8 |
(5 rows)

testdb=# select * from pg_user;
usename | usesysid | usecreatedb | usesuper | usecatupd | passwd | valuntil | useconfig
---------+----------+-------------+----------+-----------+----------+----------+-----------
gpadmin | 10 | t | t | t | ******** | |
gpmon | 19414 | t | t | t | ******** | |
(2 rows)

testdb=# \du
List of roles
Role name | Attributes | Member of
-----------+-----------------------------------+-----------
gpadmin | Superuser, Create role, Create DB |
gpmon | Superuser, Create DB |

 # 确认Performance Monitor数据库写入数据是否正常,检查是否有记录写入 「若第1次没有结果,等几秒再试一次」

[gpadmin@dbaone ~]$ psql gpperfmon -c 'SELECT * FROM system_now;'
ctime | hostname | mem_total | mem_used | mem_actual_used | mem_actual_free | swap_total | swap_used | swap_page_in | swap_page_out | cpu_user | cpu_sys | cpu_idle | load0 | load1 | load2 | quantum | disk_ro_rate | disk_wo_rate | disk_rb_rate | disk_wb
rate | netrp_rate | net_wp_rate | net_rb_rate | net_wb_rate
---------------------+----------+------------+------------+-----------------+-----------------+------------+-----------+--------------+---------------+----------+---------+----------+-------+-------+-------+---------+--------------+--------------+--------------+--------
------+-------------+-------------+-------------+-------------
2016-04-12 16:47:45 | dbatwo | 8254828544 | 8112623616 | 1895161856 | 6359666688 | 536866816 | 0 | 0 | 0 | 0.05 | 0.05 | 99.9 | 0 | 0.01 | 0 | 15 | 0 | 0 | 0 |
0 | 16 | 9 | 21129 | 20901
2016-04-12 16:47:45 | dbafour | 8254828544 | 7996710912 | 1393299456 | 6861529088 | 536866816 | 253952 | 0 | 0 | 0.07 | 0.07 | 99.83 | 0.06 | 0.06 | 0.02 | 15 | 0 | 1 | 0 |
14229 | 17 | 11 | 21225 | 41669
2016-04-12 16:47:45 | dbaone | 8254828544 | 8084590592 | 3054915584 | 5199912960 | 536866816 | 0 | 0 | 0 | 0.35 | 0.2 | 99.45 | 0 | 0 | 0 | 15 | 0 | 2 | 0 |
8715 | 22 | 17 | 2977 | 1907
2016-04-12 16:47:45 | dbathree | 8254828544 | 8094613504 | 1864966144 | 6389862400 | 536866816 | 1363968 | 0 | 0 | 0.03 | 0.05 | 99.92 | 0.04 | 0.03 | 0 | 15 | 0 | 1 | 0 |
10893 | 17 | 11 | 41906 | 20978
(4 rows)

将必要的 .pgpass 、pg_hba.conf 传到 standby master上

[gpadmin@dbaone ~]$ scp .pgpass dbafour:/home/gpadmin/
.pgpass

[gpadmin@dbaone ~]\( scp \)MASTER_DATA_DIRECTORY/pg_hba.conf dbafour:/$MASTER_DATA_DIRECTORY/pg_hba.conf
pg_hba.conf

安装 greenplum cc web

[root@dbaone greenplum-cc-web]# ./greenplum-cc-web-2.0.0-build-32-RHEL5-x86_64.bin

********************************************************************************
Do you accept the Pivotal Greenplum Database end user license
agreement? [yes | no]
********************************************************************************

yes

********************************************************************************
Provide the installation path for Greenplum Command Center or
press ENTER to accept the default installation path:     

/usr/local/greenplum-cc-web-2.0.0-build-32
********************************************************************************

/opt/greenplum/greenplum-cc-web-2.0.0-build-32

********************************************************************************
Install Greenplum Command Center into </opt/greenplum/greenplum-cc-web-2.0.0-build-32>? [yes | no]
********************************************************************************

yes

Extracting product to /opt/greenplum/greenplum-cc-web-2.0.0-build-32

********************************************************************************
Installation complete.
Greenplum Command Center is installed in /opt/greenplum/greenplum-cc-web-2.0.0-build-32

To complete the environment configuration, please ensure that the
gpcc_path.sh file is sourced.
********************************************************************************

更改权限 & 环境变量

[root]# chown gpadmin.gpadmin -R /opt/greenplum/greenplum-cc-web

[root]# chown gpadmin.gpadmin -R /opt/greenplum/greenplum-cc-web-2.0.0-build-32/

加载环境变量 gpadmin && root

vim .bashrc

# For GP web
source /opt/greenplum/greenplum-cc-web/gpcc_path.sh

source ~/.bashrc

修改权限

vim $MASTER_DATA_DIRECTORY/pg_hba.conf

 host    gpperfmon   gpmon           ::1/128    md5

$ gpstop -u

将 Master安装的,拷贝到 其他节点

$ gpccinstall -f /opt/greenplum/greenplum-db/conf/hostlist

创建 Command Center 实例

[gpadmin@dbaone ~]$ gpcmdr --setup

An instance name is used by the Greenplum Command Center as
a way to uniquely identify a Greenplum Database that has the monitoring
components installed and configured.  This name is also used to control
specific instances of the Greenplum Command Center web UI.  Instance names
can contain letters, digits and underscores and are not case sensitive.

Please enter a new instance name:
> Mr_Liu_gp
The web component of the Greenplum Command Center can connect to a
monitor database on a remote Greenplum Database.

Is the master host for the Greenplum Database remote? Yy|Nn  (default=N):
>
  The display name is shown in the web interface and does not need to be a hostname.

What would you like to use for the display name for this instance:
> greenplum_test
What port does the Greenplum Database use? (default=5432):
>
Creating instance schema in GPDB.  Please wait ...
The display name is shown in the web interface and does not need to bea hostname.

Would you like to install workload manager? Yy|Nn (default=N):        ## 若与master不在一台机器上,是不能选择y的。
>
Skipping installation of workload manager.
The Greenplum Command Center runs a small web server for the UI and web API.
This web server by default runs on port 28080, but you may specify any available port.

What port would you like the web server to use for this instance? (default=28080):
>
Users logging in to the Command Center must provide database user
credentials.  In order to protect user names and passwords, it is recommended 
that SSL be enabled.

Do you want to enable SSL for the Web API Yy|Nn (default=N):
>

Do you want to enable ipV6 for the Web API Yy|Nn (default=N):
>

Do you want to enable Cross Site Request Forgery Protection for the Web API Yy|Nn (default=N):
>

Do you want to copy the instance to a standby master host Yy|Nn (default=Y):
>
What is the hostname of the standby master host? [smdw]: dbafour
standby is dbafour

Done writing lighttpd configuration to /opt/greenplum/greenplum-cc-web/./instances/mr_liu_gp/conf/lighttpd.conf
Done writing web UI configuration to /opt/greenplum/greenplum-cc-web/./instances/mr_liu_gp/conf/gpperfmonui.conf
  Done writing web UI clustrs configuration to /opt/greenplum/greenplum-cc-web/./instances/mr_liu_gp/conf/clusters.conf

Copying instance 'mr_liu_gp' to host 'dbafour'...

Greenplum Command Center UI configuration is now complete.  If
at a later date you want to change certain parameters, you can
either re-run 'gpcmdr --setup' or edit the configuration file
located at /opt/greenplum/greenplum-cc-    web/./instances/mr_liu_gp/conf/gpperfmonui.conf.

The web UI for this instance is available at dbaone:28080/

You can now start the web UI for this instance by running: gpcmdr -- start mr_liu_gp
No instances

vim /opt/greenplum/greenplum-cc-web/www/gpmonws.py

 # 修改连接 Greenplum database 恒为真

# This service can be called without a login
# To check if GPDB is up or not
# this can be used by the GUI to decide weather to display login screen or gpstart screen
class gpdbup:
  def GET(self):
    web.header('Content-Type', 'text/xml')
    web.header('Cache-Control', 'no-store')

    cleanKrbFile()
    database_up = False

    try:
        database_up = db.is_gpdb_running(gpdb_server_name)
    except db.GPDBError, errorinfo:
        return mkerr(error.DATA_ACCESS, errorinfo.__str__())

    #return render.gpdbup(database_up)
    return render.gpdbup(True)

or

 # 修改连接 Master的主机名

# This service can be called without a login
# To check if GPDB is up or not
# this can be used by the GUI to decide weather to display login screen or gpstart screen
class gpdbup:
def GET(self):
    web.header('Content-Type', 'text/xml')
    web.header('Cache-Control', 'no-store')

    cleanKrbFile()
    database_up = False

    try:
        #database_up = db.is_gpdb_running(gpdb_server_name)
        database_up = db.is_gpdb_running('dbaone')
    except db.GPDBError, errorinfo:
        return mkerr(error.DATA_ACCESS, errorinfo.__str__())

    return render.gpdbup(database_up)

若不修改文件,则为下面状况:

启动服务

$ gpcmdr --start mr_liu_gp

dbaone:28080/