讀論文

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

看不懂

Heterogeneous Graph Transformer

Meta Relation

每條邊映射到一個三元組

Dynamic Heterogeneous Graph

把時間戳賦值到每條邊上，每個點也可以assign不同的時間戳

General GNN

從 \(l-1\) 層到 \(l\) 層：

\[H^l[t] \leftarrow \underset{\forall s \in N(t), \forall e \in E(s, t)}{\text { Aggregate }}\left(\operatorname{Extract}\left(H^{l-1}[s] ; H^{l-1}[t], e\right)\right) \]

Heterogeneous GNNs

好像要用 interaction matrices 來共享一部分 weights

HETEROGENEOUS GRAPH TRANSFORMER

Its idea is to use the meta relations of heterogeneous graphs to parameterize weight matrices for the heterogeneous mutual attention, message passing, and propagation steps.

general attention-based GNNs

\[H^l[t] \leftarrow \underset{\forall s \in N(t), \forall e \in E(s, t)}{\text { Aggregate }}(\text { Attention }(s, t) \cdot \text { Message }(s)) \]

\[\begin{aligned} \operatorname{Attention}_{G A T}(s, t) & =\underset{\forall s \in N(t)}{\operatorname{Softmax}}\left(\vec{a}\left(W H^{l-1}[t] \| W H^{l-1}[s]\right)\right) \\ \operatorname{Message}_{G A T}(s) & =W H^{l-1}[s] \\ \operatorname{Aggregate}_{G A T}(\cdot) & =\sigma(\operatorname{Mean}(\cdot)) \end{aligned} \]

HGT計算方式

注意他矩陣W都是type-aware的

\[\begin{gathered} \operatorname{Head}_k^{A T T}(i, j)=\left(\frac{\mathbf{K}_i^k \mathbf{W}_{\psi(i, j)}^{A T T} \mathbf{Q}_j^{k^{\mathrm{T}}}}{\sqrtw0obha2h00}\right) \mu(\phi(i), \psi(i, j), \phi(j)) \\ \operatorname{Attention}(i, j)=\operatorname{Softmax}_{i \in N(j)}\left(\|_k \operatorname{Head}_k^{A T T}(i, j)\right) \end{gathered} \]

\[\begin{gathered} \operatorname{Message}(i, j)=\|_k \mathbf{W}_{\phi(i)}^k \mathbf{h}_i \mathbf{W}_{\psi(i, j)}^{M S G} \\ \mathbf{h}_j=\sum_{i \in N(j)} \operatorname{Attention}(i, j) \odot \operatorname{Message}(i, j) \end{gathered} \]

RTE

就是根據一條邊時間差 \(\Delta t\) 給sourse node加上一個生成的向量（由一堆正余弦函數作為基底，過一遍linear）

HGSampling

對于webscale的圖需要進行sample，大概是通過當前點擴展鄰居節點，對于每個type分別算概率

似乎每種點都取了n個，沒有考慮數量差別

A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources

Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications

Network Schema: 一個paradigm

Metapath：元路徑

Heterogeneous Graph Embedding

convolution-based HGNN

思想是把周圍點的信息aggregate到當前點，形成embedding

更高效但是參數更多空間消耗更大

HAN

在一個metapath的相鄰節點之中，attention和embedding相互計算

獲得多個metapath的 different semantic node embeddings 之后，再過一遍（mlp，激活函數，attention vector）計算attention，再計算最后的embedding

HAHE：uses cosine similarity instead of attention mechanism to calculate the two kinds of importance

MAGNN:把metapath的中間節點也用encoder存進了semantic信息，不過他encoder到底怎么工作的沒仔細說不太懂

這幾種都要人工設置metapath

GTN

Graph Transformer Networks，分出subgraphs然后學習embedding

But GTN only considers edge types with ignoring diferent types of nodes.

HetSANN

Heterogeneous Graph Structural Attention Neural Network

首先把neighbor nodes映射進選中的點的空間，然后通過type-aware attention layer來學習每個點的embedding，這樣可以考慮周圍點的類型不同以及重要性不同

HGT

同論文

HetSANN和HGT都使用分層注意力機制來代替metapath，但是生成了更多的參數

HGCN

認為異質圖是由多個二分圖子圖構成的，在每個子圖上用GCN，再用attention（type-aware）aggregate起來獲得最終的embedding

Autoencoder?based approaches

HIN2Vec

conceptualNN 本來是想算出i和j對每種關系的概率，但是成本過高，于是嘗試算出i和j有特定關系r的概率

\[P(r|i,j) = sigmoid\left(\sum\mathbf{W}_I\vec i \cdot \mathbf{W}_J\vec j\cdot\mathbf{W}_R\vec r\right) \]

然后計算交叉熵損失

訓練數據生成是用隨機游走，生成metapath，一種metapath代表一種關系

SHINE

用三個autoencoder計算三種不同的語義信息

utilizes the topological structure of heterogeneous graphs

HNE

每個點代表text或者image，adopts CNN and fully connected layers

NSHE

首先使用GCN（還是先project成向量，然后aggregate）來涵蓋一階proximity，然后使用多個autoencoder最后拼起來進行預測任務

Adversarial?based approaches

對抗神經網絡，用一個discriminator和一個generator互相學習

HEGAN

discriminator:

\[D\left(\mathbf{h}_j \mid i, r ; \theta^D\right)=\frac{1}{1+\exp \left(-\mathbf{h}_i^D \mathbf{M}_r^D \mathbf{h}_j^D\right)} \]

generator:

\[G\left(i, r ; \theta^G\right)=\sigma\left(\mathbf{W}_L \cdots \sigma\left(\mathbf{W}_1 \mathbf{h}+\mathbf{b}_1\right)+\mathbf{b}_L\right) \]

對于一個關系，每次generator生成一個fakenode，然后給discriminator進行判別，計算各自loss，如果不能分辨那么generator就可以拿來生成embedding

Dynamic heterogeneous graph learning

對每個時刻算一個embedding，然后算attention加到一起

posted @ 2023-08-26 13:26 lcyfrog 閱讀(146) 評論(0) 收藏舉報

刷新頁面返回頂部

lcyfrog

獨りぼっちの　作戦だ

讀論文

讀論文

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Heterogeneous Graph Transformer

Meta Relation

Dynamic Heterogeneous Graph

General GNN

Heterogeneous GNNs

HETEROGENEOUS GRAPH TRANSFORMER

general attention-based GNNs

HGT計算方式

RTE

HGSampling

A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources

Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications

convolution-based HGNN

HAN

GTN

HetSANN

HGT

HGCN

Autoencoder?based approaches

HIN2Vec

SHINE

HNE

NSHE

Adversarial?based approaches

HEGAN

Dynamic heterogeneous graph learning

公告

lcyfrog

獨りぼっちの 作戦だ

讀論文

讀論文

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Heterogeneous Graph Transformer

Meta Relation

Dynamic Heterogeneous Graph

General GNN

Heterogeneous GNNs

HETEROGENEOUS GRAPH TRANSFORMER

general attention-based GNNs

HGT計算方式

RTE

HGSampling

A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources

Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications

convolution-based HGNN

HAN

GTN

HetSANN

HGT

HGCN

Autoencoder?based approaches

HIN2Vec

SHINE

HNE

NSHE

Adversarial?based approaches

HEGAN

Dynamic heterogeneous graph learning

公告

獨りぼっちの　作戦だ