neo4j實(shí)現(xiàn)疾病知識(shí)圖譜實(shí)戰(zhàn)

neo4j、知識(shí)圖譜、python、csv輸入、查詢

neo4j實(shí)現(xiàn)疾病知識(shí)圖譜實(shí)戰(zhàn)

1. neo4j安裝

linux下安裝，直接創(chuàng)建腳本Neo4j_setup.sh安裝腳本，執(zhí)行安裝即可，安裝完成后，打開(kāi)瀏覽器http:// localhost:7474，默認(rèn)用戶名/密碼為neo4j/neo4j，首次登錄需要修改密碼

#!/bin/bash

#neo4j 安裝

#1)設(shè)置hosts綁定

IP=`ifconfig|sed -n 2p|awk '{print $2}'|cut -d ":" -f2`

echo "$IP neo4j" >>/etc/hosts

#2)下載安裝neo4j

cd /home/tools

wget -c https://neo4j.com/artifact.php?name=neo4j-community-3.4.14-unix.tar.gz

tar zxvf artifact.php\?name\=neo4j-community-3.4.14-unix.tar.gz -C /usr/local/

ln -s /usr/local/neo4j-community-3.4.14 /usr/local/neo4j-community

#3)配置環(huán)境變量

cat >/etc/profile.d/neo4j <<EOF

export NEO4J_HOME=/usr/local/neo4j

export PATH=\$PATH:\$NEO4J_HOME/bin

EOF

source /etc/profile.d/neo4j

#4) 配置資源

sed -i 's/#dbms.memory.heap.initial_size=512m/dbms.memory.heap.initial_size=2048m/g' /usr/local/neo4j-community/conf/neo4j.conf

sed -i 's/#dbms.memory.heap.max_size=512m/dbms.memory.heap.max_size=2048m/g' /usr/local/neo4j-community/conf/neo4j.conf

sed -i 's/#dbms.connectors.default_listen_address=0.0.0.0/dbms.connectors.default_listen_address=neo4j/g' /usr/local/neo4j-community/conf/neo4j.conf

#5) 配置neo4j啟動(dòng)腳本

cat >/etc/init.d/neo4j <<EOF

#!/bin/bash

### BEGIN REDHAT INFO

# chkconfig: 2345 99 20

# description: Neo4j Graph Database server

SCRIPTNAME=\$0

NEO4J_CONF=/usr/local/neo4j-community/conf

NEO4J_HOME=/usr/local/neo4j-community

NEO_USER=root

NEO4J_ULIMIT_NOFILE=60000

PATH=/sbin:/usr/sbin:/bin:/usr/bin

NAME=neo4j

DAEMON=\${NEO4J_HOME}/bin/\${NAME}

PIDDIR=\${NEO4J_HOME}/run

PIDFILE=\${PIDDIR}/neo4j.pid

SCRIPTNAME=/etc/init.d/\${NAME}

SYSTEMCTL_SKIP_REDIRECT=1

[ -x "\$DAEMON" ] || exit 0

#[ -r \${NEO4J_CONF}/\${NAME}.conf ] && . \${NEO4J_CONF}/\${NAME}.conf

[ -n "\${NEO_USER}" ] || NEO_USER=\${NAME}

# Debian distros and SUSE

has_lsb_init()

{

  test -f "/lib/lsb/init-functions"

}

# RedHat/Centos distros

has_init()

{

  test -f "/etc/init.d/functions"

}

if has_lsb_init ; then

  . /lib/lsb/init-functions

elif has_init ; then

  . /etc/init.d/functions

else

  echo "Error: your platform is not supported by \${NAME}" >&2

  exit 1

fi

do_start()

{

  do_ulimit

  [ -d "\${PIDDIR}" ] || mkdir -p "\${PIDDIR}"

  chown "\${NEO_USER}:" "\${PIDDIR}"

  if has_lsb_init ; then

    start-stop-daemon --chuid \${NEO_USER} --start --quiet --oknodo --pidfile \${PIDFILE} --exec \${DAEMON} -- start

  else

    daemon --user="\${NEO_USER}" --pidfile="\${PIDFILE}" "\${DAEMON} start > /dev/null 2>&1 &"

  fi

}

do_stop()

{

  \${DAEMON} stop

}

do_status()

{

  if has_lsb_init ; then

    status_of_proc -p "\${PIDFILE}" "\${DAEMON}" "\${NAME}"

  else

    status -p "\${PIDFILE}" "\${NAME}"

  fi

}

do_ulimit()

{

  if [ -n "\${NEO4J_ULIMIT_NOFILE}" ]; then

    ulimit -n "\${NEO4J_ULIMIT_NOFILE}"

  fi

}

case "\$1" in

  start)

    do_start

    ;;

  stop)                                                         

    do_stop

    ;;

  status)

    do_status

    ;;

  restart|force-reload)

    do_stop && do_start

    ;;

  *)

    echo "Usage: \$SCRIPTNAME {start|stop|status|restart|force-reload}" >&2

    exit 3

    ;;

esac

EOF

#6) 設(shè)置權(quán)限

chmod +x /etc/init.d/neo4j

#7) 啟動(dòng)neo4j

service neo4j start

#8) 配置開(kāi)機(jī)自啟動(dòng)

chkconfig neo4j on

echo 'Neo4j install done'

2. neo4j圖數(shù)據(jù)庫(kù)簡(jiǎn)介

Neo4j是一款是由java語(yǔ)言實(shí)現(xiàn)的圖數(shù)據(jù)庫(kù)，圖形數(shù)據(jù)庫(kù)將數(shù)據(jù)以圖的數(shù)據(jù)結(jié)構(gòu)進(jìn)行存儲(chǔ)和管理，并且能夠以高度可問(wèn)的方式優(yōu)雅地表示任何種類的數(shù)據(jù)，而Neo4j是基于屬性圖模型（Property Graph Model）的數(shù)據(jù)庫(kù)

在屬性圖中存在如下元素:

1、實(shí)體(Entity)

??a) 節(jié)點(diǎn)（Node）

??b) 關(guān)系（Relationship）

2、邊/路徑(Path)

3、記號(hào)（Token）

??a) 標(biāo)簽(Label)

??b) 關(guān)系類型(Relationship Type)

??c) 屬性key（Property Key）

4、屬性（Property）

參考http://www.rzrgm.cn/jpfss/p/11268835.html

3. neo4j基本語(yǔ)法

3.1 Cypher

neo4j的查詢語(yǔ)言為Cypher，是一個(gè)描述性的圖形查詢語(yǔ)言

說(shuō)明：()內(nèi)代表節(jié)點(diǎn)，[]代表關(guān)系，->關(guān)系方向，{}代表屬性，:后面跟記號(hào)如節(jié)點(diǎn)的標(biāo)簽、關(guān)系的類型

節(jié)點(diǎn)：(Variable:Lable{Key1:Value1,Key2,Value2,...})

關(guān)系：[Variable:RelationshipType{Key1:Value1,Key2:Value2,...}]

3.2 語(yǔ)法

l 新節(jié)點(diǎn)、新關(guān)系、無(wú)屬性

create ()-[]->()

l 新節(jié)點(diǎn)、新關(guān)系、有屬性

create (:{})-[:{}]->(:{})

l 已有節(jié)點(diǎn)、新關(guān)系、無(wú)屬性

MATCH (:),(:) create ()-[:]->()

先用match找到兩個(gè)節(jié)點(diǎn)，再給節(jié)點(diǎn)添加關(guān)系（如果不用match，則會(huì)新建節(jié)點(diǎn)）

另外，同時(shí)執(zhí)行時(shí)（一個(gè)分號(hào)內(nèi)），前面節(jié)點(diǎn)會(huì)在新建關(guān)系時(shí)被識(shí)別（不用match），否則，會(huì)認(rèn)為是新的節(jié)點(diǎn)

新節(jié)點(diǎn)可以與已有節(jié)點(diǎn)名稱、標(biāo)簽、屬性都相同（如同年同月同日生同名同性別的人），但是會(huì)自動(dòng)生成唯一標(biāo)識(shí)id以區(qū)分

l merge

merge(:{})

可以看成是match和create的合體，找不到則創(chuàng)建節(jié)點(diǎn)，找到則更新節(jié)點(diǎn)

l 同時(shí)匹配兩標(biāo)簽

match (n) where any(label in labels(n) WHERE label in ['label1', 'label2']) return n

4. 實(shí)戰(zhàn)應(yīng)用

4.1 診斷歸一知識(shí)圖譜

create (disease1:頂級(jí)節(jié)點(diǎn):diagnosis{name:'疾病名稱'})

create (disease2:頂級(jí)節(jié)點(diǎn):diagnosis{name:'呼吸系統(tǒng)疾病名稱'})

create (disease2)-[:belong_to]->(disease1)

 

create (standard01:標(biāo)準(zhǔn)詞:diagnosis{name:'間質(zhì)性肺疾病'})

create (standard02:標(biāo)準(zhǔn)詞:diagnosis{name:'矽肺'})

create (standard1:標(biāo)準(zhǔn)詞:diagnosis{name:'矽肺[硅肺]壹期'})

create (standard2:標(biāo)準(zhǔn)詞:diagnosis{name:'矽肺[硅肺]貳期'})

create (standard3:標(biāo)準(zhǔn)詞:diagnosis{name:'矽肺[硅肺]叁期'})

create (standard01)-[:belong_to]->(disease2)

create (standard02)-[:belong_to]->(standard01)

create (standard1)-[:belong_to]->(standard02)

create (standard2)-[:belong_to]->(standard02)

create (standard3)-[:belong_to]->(standard02)

 

create (origin1:原始詞:diagnosis{name:'硅肺'})

create (origin2:原始詞:diagnosis{name:'硅沉著肺'})

create (origin3:原始詞:diagnosis{name:'矽肺[硅沉著病]'})

create (origin4:原始詞:diagnosis{name:'矽肺(硅沉著病)'})

create (origin5:原始詞:diagnosis{name:'矽肺(硅肺)'})

create (origin6:原始詞:diagnosis{name:'矽肺Ⅰ期'})

create (origin7:原始詞:diagnosis{name:'矽肺(硅肺)I期'})

create (origin8:原始詞:diagnosis{name:'矽肺(I期)'})

create (origin9:原始詞:diagnosis{name:'矽肺(II期)'})

create (origin10:原始詞:diagnosis{name:'矽肺(硅肺)Ⅱ期'})

create (origin11:原始詞:diagnosis{name:'矽肺(硅肺)Ⅲ期'})

create (origin1)-[:standardized]->(standard02)

create (origin2)-[:standardized]->(standard02)

create (origin3)-[:standardized]->(standard02)

create (origin4)-[:standardized]->(standard02)

create (origin5)-[:standardized]->(standard02)

create (origin6)-[:standardized]->(standard1)

create (origin7)-[:standardized]->(standard1)

create (origin8)-[:standardized]->(standard1)

create (origin9)-[:standardized]->(standard2)

create (origin10)-[:standardized]->(standard2)

create (origin11)-[:standardized]->(standard3)

4.2 圖形效果

5. Python實(shí)現(xiàn)輸入與查詢

5.1 Python環(huán)境

Anaconda官網(wǎng)下載安裝即可，Anaconda包含了conda、Python在內(nèi)的超過(guò)180個(gè)科學(xué)包及其依賴項(xiàng)，內(nèi)置spyder、jupyter調(diào)試工具

5.2 讀取csv/excel

# -*- coding: utf-8 -*-

"""

Created on Wed Sep 30 10:29:49 2020

@author:Quentin

"""

 

from py2neo import Graph, Node, Relationship,NodeMatcher

import pandas as pd

import re

import os

import sys

 

class CreateGraph:

    def __init__(self,csv_name):

        #當(dāng)前目錄

        cur_dir = '/'.join(os.path.abspath('__file__').split('/')[:-1])

        self.data_path = os.path.join(cur_dir, csv_name)

        self.graph = Graph("http://192.168.31.240:7474", username="neo4j", password="123456")

                 

    def read_file(self):

        all_data = pd.read_csv(self.data_path, encoding='utf-8').loc[:, :].values

        return all_data    

           

    def create_graph(self):

        all_data = self.read_file()

        top_node = 'undefined'

        matcher = NodeMatcher(self.graph)

        if (all_data[0][1] == '頂級(jí)節(jié)點(diǎn)'):

            top_node = all_data[0][0]

           

        #創(chuàng)建節(jié)點(diǎn)

        for row_data in all_data:

            #判斷node是否存在

            node_match =  matcher.match(row_data[1],name = row_data[0],topNode = top_node).first()

            if node_match is None:

                node = Node(row_data[1],name = row_data[0],topNode = top_node)

                self.graph.create(node)

                print('創(chuàng)建新節(jié)點(diǎn)：' + str(node).encode('utf-8').decode('unicode_escape'))

                

        #創(chuàng)建關(guān)系

        for row_data in all_data:

            if len(str(row_data[2])) > 0 and str(row_data[2]) != 'nan':

                node1 = matcher.match(row_data[1],name = row_data[0],topNode = top_node).first()

                node2 = self.node_std_or_top(matcher,row_data[2],top_node)

                if  node1 is not None and node2 is not None:                   

                    if str(row_data[1]) == '原始詞':

                        relation = Relationship(node1,'standard',node2)

                    else:

                        relation = Relationship(node1,'belong_to',node2)

                    self.graph.create(relation)

                    print('創(chuàng)建關(guān)系：' + str(relation))

 

    def node_std_or_top(self,matcher,name,topNode):

        node = matcher.match('標(biāo)準(zhǔn)詞',name = name,topNode = topNode).first()

        if  node is  None :  

            node = matcher.match('頂級(jí)節(jié)點(diǎn)',name = name,topNode = topNode).first()

        return node

  

       

if __name__ == "__main__":

    str_csv = sys.argv[1]

    handler = CreateGraph(str_csv)

    handler.create_graph()

5.3 詞表查找

# -*- coding: utf-8 -*-

"""

Created on Wed Sep 30 10:29:49 2020

@author: Quentin

"""

 

from py2neo import Graph, Node, Relationship,NodeMatcher

import pandas as pd

import re

import os

import sys

 

class SelectStandard:

    def __init__(self):

        self.graph = Graph("http://192.168.31.240:7474", username="neo4j", password="123456")

         

    #查詢上級(jí)詞（標(biāo)準(zhǔn)詞）

    def select_upper_vocab(self,orig,top_node='',label='原始詞'):

        """   

        查找輸入詞的上級(jí)節(jié)點(diǎn)

        Parameters

        ----------

        orig : 輸入詞

             原始詞、標(biāo)準(zhǔn)詞都可

        top_node : 頂級(jí)節(jié)點(diǎn), optional

             The default is ''.

        label : label類型，原始詞、標(biāo)準(zhǔn)詞, optional

             The default is '原始詞'.

            

        Returns

        -------

        返回上級(jí)節(jié)點(diǎn)，字符型

        """

        if top_node == '':

            query = "match(n:%s)-[r]->(m) where n.name = '%s' return m.name" %(label,orig)

            result = self.graph.run(query).to_ndarray()

        else:

            query = "match(n:%s)-[r]->(m) where n.name = '%s' and n.topNode = '%s' return m.name" %(label,orig,top_node)

            result = self.graph.run(query).to_ndarray()           

        if len(result) > 0:

            return result[0][0]

        else:

            return '無(wú)'

   

    #查詢同級(jí)詞（原始詞）

    def select_equal_vocab(self,orig,top_node='',label='原始詞'):

        """

        查找輸入詞的同級(jí)節(jié)點(diǎn)

 

        Parameters

        ----------

        orig : 輸入詞

            原始詞、標(biāo)準(zhǔn)詞都可

        top_node : 頂級(jí)節(jié)點(diǎn), optional

             The default is ''.

        label : label類型，原始詞、標(biāo)準(zhǔn)詞, optional

             The default is '原始詞'.

 

        Returns

        -------

        返回同級(jí)節(jié)點(diǎn)，數(shù)組

 

        """

        if top_node == '':

            query = "match(n1:%s)-[r1]->(m1) where n1.name ='%s' match(n2:%s)-[r2]->(m1) where n2.name <> '%s'  return n2.name" %(label,orig,label,orig)

            result = self.graph.run(query).to_ndarray()

        else:

            query = "match(n1:%s)-[r1]->(m1) where n1.name ='%s' and n1.topNode = '%s' match(n2:%s)-[r2]->(m1) where n2.name <> '%s' and n2.topNode = '%s' return n2.name" %(label,orig,top_node,label,orig,top_node)

            result = self.graph.run(query).to_ndarray()

        if len(result) > 0:

            rs_arr = []

            for rs in result:

                rs_arr.append(rs[0])

            return rs_arr

        else:

            return '無(wú)'

   

if __name__ == "__main__":

    #輸入?yún)?shù)

    str_vocab = sys.argv

    upper_vocab_param = ['','','原始詞']

    equal_vocab_param = ['','','原始詞']

    for i in range(0,min(3,len(str_vocab)-1)):

        upper_vocab_param[i] = str_vocab[i+1]

        equal_vocab_param[i] = str_vocab[i+1]

    #輸出結(jié)果

    handler = SelectStandard()

    upper_vocab = handler.select_upper_vocab(upper_vocab_param[0],upper_vocab_param[1],upper_vocab_param[2])

    print("標(biāo)準(zhǔn)詞：")

    print(upper_vocab)

    equal_vocab = handler.select_equal_vocab(equal_vocab_param[0],equal_vocab_param[1],equal_vocab_param[2])

    print("同義詞：")

    print(equal_vocab)

5.4 輸出結(jié)果

posted on 2020-09-30 17:26 ohmyuan 閱讀(2420) 評(píng)論(0) 收藏舉報(bào)

刷新頁(yè)面返回頂部

導(dǎo)航

neo4j實(shí)現(xiàn)疾病知識(shí)圖譜實(shí)戰(zhàn)

neo4j實(shí)現(xiàn)疾病知識(shí)圖譜實(shí)戰(zhàn)

1. neo4j安裝

2. neo4j圖數(shù)據(jù)庫(kù)簡(jiǎn)介

3. neo4j基本語(yǔ)法

3.1 Cypher

3.2 語(yǔ)法

4. 實(shí)戰(zhàn)應(yīng)用

4.1 診斷歸一知識(shí)圖譜

4.2 圖形效果

5. Python實(shí)現(xiàn)輸入與查詢

5.1 Python環(huán)境

5.2 讀取csv/excel

5.3 詞表查找

5.4 輸出結(jié)果