6 AutoML

• AutoMLQuickStart - C# console application that shows how to get started with the AutoML API.
• AutoMLAdvanced - C# console application that shows the following concepts:
  • Modifying column inference results
  • Excluding trainers
  • Configuring monitoring
  • Choosing tuners
  • Cancelling experiments
• AutoMLEstimators - C# console application that shows how to:
  • Customize search spaces
  • Create sweepable estimators
• AutoMLTrialRunner - C# console application that shows how to create your own trial runner for the Text Classification API.

7 Natural Language Processing (NLP)

• TextClassification - C# console app that shows how to use the Text Classification API for inferencing using code generated by Model Builder. The model is trained using Model Builder.
• TextClassification_Sentiment_Razor - ASP.NET Core Razor Pages application for sentiment analysis. Code sample for Analyze sentiment of website comments in a web application using ML.NET Model Builder tutorial. Model is trained using Model Builder.
• SentenceSimilarity - C# console app that shows how to use the Sentence Similarity API. Like the Text Classification API, the Sentence Similarity API uses a NAS-BERT transformer-based deep learning model built with TorchSharp to compare how similar two pieces of text are.

8 例子解析

數據來源，從這個地址下載

• Taxi Fare
• Yelp Reviews (yelp_labelled.txt)
• Home Depot

9 句子相似度 SentenceSimilarity

【測試時機器沒有cuda環境，使用cpu進行訓練】
train.csv - 訓練集，包含產品、搜索和相關性分數

home-depot-sentence-similarity.csv數據代碼庫沒有，原始的train.csv 和 home-depot-sentence-similarity.csv關系，可以參考如下下載和生成
https://github.com/dotnet/machinelearning-samples/issues/982 【我按照代碼定義的格式寫了合并 csv的數據預處理，如下
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Transforms;
namespace SentenceSimilarity
{
internal class GenData
{
// id product_uid product_title search_term relevance
// 2 100001 Simpson Strong-Tie 12-Gauge Angle angle bracket 3
public class HomeDepot
{
[LoadColumn(0)]
public int id { get; set; }
[LoadColumn(1)]
public int product_uid { get; set; }
[LoadColumn(2)]
public string product_title { get; set; }
[LoadColumn(3)]
public string search_term { get; set; }
[LoadColumn(4)]
public string relevance { get; set; }
}
// https://learn.microsoft.com/en-us/dotnet/api/microsoft.ml.custommappingcatalog.custommapping?view=ml-dotnet
[CustomMappingFactoryAttribute("product_description")]
private class ProdDescCustomAction : CustomMappingFactory<HomeDepot, CustomMappingOutput>
{
// We define the custom mapping between input and output rows that will
// be applied by the transformation.
public static void CustomAction(HomeDepot input, CustomMappingOutput
output) => output.product_description = prodDesc[input.product_uid.ToString()];
public override Action<HomeDepot, CustomMappingOutput> GetMapping()
=> CustomAction;
}
// Defines only the column to be generated by the custom mapping
// transformation in addition to the columns already present.
private class CustomMappingOutput
{
public string product_description { get; set; }
}
static Dictionary<string, string> prodDesc = new Dictionary<string, string>();
static void Main(string[] args)
{
var mlContext = new MLContext(seed: 1);
var DataPath = Path.GetFullPath(@"........\Data\product_descriptions.csv");
{
IDataView dv = mlContext.Data.LoadFromTextFile(DataPath, hasHeader: true, separatorChar: ',', allowQuoting: true,
columns: new[] {
new TextLoader.Column("product_uid",DataKind.String,0),
new TextLoader.Column("product_description",DataKind.String,1)
}
);
foreach (var row in dv.Preview(maxRows: 15_0000).RowView)
{
string uid="", desc="";
foreach (KeyValuePair<string, object> column in row.Values)
{
if (column.Key == "product_uid")
{
uid = column.Value.ToString();
}
else
{
desc= column.Value.ToString();
}
}
prodDesc[uid] = desc;
}
}
DataPath = Path.GetFullPath(@"........\Data\train.csv");
IDataView dataView = mlContext.Data.LoadFromTextFile(DataPath, hasHeader: true, separatorChar: ',', allowQuoting: true);
var preViewTransformedData = dataView.Preview(maxRows: 5);
foreach (var row in preViewTransformedData.RowView)
{
var ColumnCollection = row.Values;
string lineToPrint = "Row--> ";
foreach (KeyValuePair<string, object> column in ColumnCollection)
{
lineToPrint += $"| {column.Key}:{column.Value}";
}
Console.WriteLine(lineToPrint + "\n");
}
var pipeline = mlContext.Transforms.CustomMapping(new ProdDescCustomAction().GetMapping(), contractName: "product_description");
var transformedData = pipeline.Fit(dataView).Transform(dataView);
//mlContext.ComponentCatalog.RegisterAssembly(typeof(IsUnderThirtyCustomAction).Assembly);
Console.WriteLine("save file");
using FileStream fs = new FileStream(Path.GetFullPath(@"........\Data\home-depot-sentence-similarity.csv"), FileMode.Create);
mlContext.Data.SaveAsText(transformedData, fs, schema: false, separatorChar:',');
}
}
}
具體參考 https://gitee.com/iamops/x-unix-dotnet/blob/main/ml.net2/SentenceSimilarity/GenData.cs
】

數據放好后運行時，會類似如下下載模型文件：
[Source=NasBertTrainer; TrainModel, Kind=Trace] Channel started
[Source=NasBertTrainer; Ensuring model file is present., Kind=Trace] Channel started
[Source=NasBertTrainer; Ensuring model file is present., Kind=Info] Downloading NasBert2000000.tsm from https://aka.ms/mlnet-resources/models/NasBert2000000.tsm to C:\Users\homelap\AppData\Local\Temp\mlnet\NasBert2000000.tsm
[Source=NasBertTrainer; Ensuring model file is present., Kind=Info] NasBert2000000.tsm: Downloaded 3620 bytes out of 17907563
...

TorchSharp目前版本沒有正式發布，例子運行問題多多，如上步驟放好數據文件后，直接運行出現
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Repeat 2 times:

at TorchSharp.torch+random.THSGenerator_manual_seed(Int64)

at TorchSharp.torch+random.manual_seed(Int64) ...錯誤

https://github.com/dotnet/machinelearning/issues/6669 按照這個說明設置也不對

按如上的設置版本也不對，仍是運行異常，那就進入TorchSharp的源碼看看吧。

10 TorchSharp

初步的軟件庫結構

如上可見，在Microsoft.ML的大框架下【Microsoft.ML.Core.dll Microsoft.ML.dll Microsoft.ML.PCA.dll Microsoft.ML.Transforms.dll Microsoft.ML.Data.dll Microsoft.ML.KMeansClustering.dll Microsoft.ML.StandardTrainers.dll】
針對Torch，按照ML的框架結構，擴展了Microsoft.ML.TorchSharp這層【Microsoft.ML.TorchSharp.dll】，如下是其擴展的概覽圖

針對CPU/GPU 的場景，分別提供不同的庫支持
TorchSharp TorchAudio TorchVision是使用C#語言實現的不同業務類別的庫，這個庫在pytorch的C語言庫的基礎上進行成抽象封裝，為Microsoft.ML.TorchSharp提供服務；該工程使用C++語言提供了LibTorchSharp庫【

】，供TorchSharp/TorchAudio/TorchVision來調用
LibTorchSharp最后使用P/Invoke的模式來調用 pytorch的c語言庫
如下就是TorchSharp 工程封裝C語言和打包的配置

相關工程的參考地址：
https://github.com/dotnet/TorchSharp
https://github.com/dotnet/TorchSharpExamples
https://www.nuget.org/packages/TorchSharp/
As we build up to a v1.0 release, we will continue to make breaking changes, but only when we consider it necessary for usability. Similarity to the PyTorch experience is a primary design tenet, and we will continue on that path.
如上可見TorchSharp由于未發布1.0,因此接口變化很快，而且官方倉庫在nuget發布的的也沒有分支和tag，兼容性問題較大
比如找到這個tag 如 https://github.com/dotnet/TorchSharpExamples/releases/tag/v0.95.4 這個的官方例子運行都有問題【
TorchSharpExamples-0.95.4.tar\TorchSharpExamples-0.95.4\src\CSharp\CSharpExamples

random.manual_seed(1); 這個直接不能訪問，報異常
但取main分支，0.100.5的版本運行正常




】

回到當前的例子：

TorchSharp.dll!TorchSharp.torch.random.manual_seed(long seed) Line 21623
at TorchSharp\torch.cs(21623)

0.98.1的版本在 https://github.com/dotnet/TorchSharp 這個倉庫沒有分支和標簽，這代碼夠亂的， https://www.nuget.org/packages/TorchSharp-cpu 不知道這個倉庫發布的版本的代碼來自哪里？
0.99.3的版本 https://github.com/dotnet/TorchSharp/releases/tag/v0.99.3 和Microsoft.ML.TorchSharp版本直接不兼容，運行缺少部分實現，估計版本迭代節奏差別大
找了個有分支的代碼0.98.2,分析下

11 TorchSharp 0.98.2版本調試

https://github.com/dotnet/TorchSharp/tree/0.98.2 整個這個分支跟蹤下，按照其DEVGUIDE.md的說明，使用devenv構建

msbuild TorchSharp.sln

msbuild TorchSharp.sln /p:Configuration=Release /p:Platform=x64

libtorch 的版本和 pytorch 是對應的，比如 libtorch 1.6.0 對應于 pytorch 1.6.0。

• DEBUG模式版本

https://download.pytorch.org/libtorch/cpu/
https://download.pytorch.org/libtorch/cu113
https://download.pytorch.org/libtorch/cu113/libtorch-win-shared-with-deps-debug-1.11.0%2Bcu113.zip
https://download.pytorch.org/libtorch/cpu/libtorch-win-shared-with-deps-debug-1.11.0%2Bcpu.zip
下載后會將這些編譯結果自動下載下來

libtorch-cpu\libtorch-win-shared-with-deps-debug-1.11.0%2Bcpu.zip 650M
libtorch-cuda-11.3\libtorch-win-shared-with-deps-debug-1.11.0%2Bcu113.zip 2.7G
文件很大
libtorch-cpu\libtorch-win-shared-with-deps-1.11.0%2Bcpu.zip 143M
libtorch-cuda-11.3\libtorch-win-shared-with-deps-1.11.0%2Bcu113.zip 2G
準備好后，直接vs中調試出現問題的函數，正常沒問題

將例子代碼拿過來，運行，也正常

• Release模式版本

https://download.pytorch.org/libtorch/cu113/libtorch-win-shared-with-deps-1.11.0%2Bcu113.zip
https://download.pytorch.org/libtorch/cpu/libtorch-win-shared-with-deps-1.11.0%2Bcpu.zip

12 例子正常運行

經嘗試，只要將如下libtorch的相關庫：

例子默認的發布文件不能工作，更換后如下

初步估計是nuget中發布0.98.2版本可能哪里有不一致的地方
【具體的工程參考
https://gitee.com/iamops/x-unix-dotnet/blob/main/ml.net2/SentenceSimilarity/SentenceSimilarity.csproj 】
運行過程

13 小結

Torch的數據訓練使用cpu進行速度的確很慢

SentenceSimilarity -官方的這個例子由于混合了c#和pytorch的c版本，由于torch這塊的版本不穩定，使用比較麻煩。具體原因上述已分析。