Scala自定義傳參
最近在寫Spark導數程序的時候,為了實現程序的多元化,即使用同一套程序,實現不同場景的需求,而參考Spark的org.apache.spark.deploy.master.MasterArguments解析main()方法參數寫法
@tailrec
def parameterMatching(args: List[String]): Unit={
val argsList: List[String] = args.toList
argsList match {
case ("--inputPath" | "-i") :: value :: tail =>
hdfsPath = value
parameterMatching(tail)
case ("--execSQL" | "-e") :: value :: tail =>
execSQL = value
parameterMatching(tail)
case ("--outputSchema" | "-s") :: value :: tail =>
outputSchema = value
parameterMatching(tail)
case ("--headLine" | "-h") :: value :: tail =>
headLine = value
parameterMatching(tail)
case ("--writeMode" | "-m") :: value :: tail =>
writeMode = value
parameterMatching(tail)
case ("--outputPath" | "-o") :: value :: tail =>
outputPath = value
parameterMatching(tail)
case ("--numPartitions" | "-n") :: value :: tail =>
numPartitions = value.toInt
parameterMatching(tail)
case ("--help") :: tail =>
printUsageAndExit(0)
case Nil => // No-op
case _ =>
printUsageAndExit(1)
}
}
def printUsageAndExit(exitCode: Int): Unit ={
System.err.println(
"Usage: Hdfs2MppAbsolutePath [options]\n" +
"\n" +
"Options:\n" +
" -i hdfsPath, --inputPath hdfsPath 讀HDFS文件路徑 (必選。)\n" +
" -e execSQL, --execSQL execSQL 對讀數據執行的SQL (可選。default: select * from temp)\n" +
" -s outputSchema, --outputSchema outputSchema 寫數據的格式 (可選。default: csv)\n" +
" -h headLine, --headLine headLine 是否打印頭標題。(可選。default: true)\n" +
" -m writeMode, --writeMode writeMode 寫入數據模式。 (可選。default: append)\n" +
" -o outputPath, --outputPath outputPath 寫到本地路徑 (必選。)\n" +
" -n numPartitions, --numPartitions numPartitions 設置寫入本地的分區數 (可選。default 1)\n"
)
// scalastyle:on println
System.exit(exitCode)
}
屆時main()方法內部只要調用這個方法舊可以實現程序更具不同場景,做出不同的導出樣式。
def main(args: Array[String]): Unit = {
val argsList: List[String] = args.toList
println("\n 執行參數匹配:")
parameterMatching(argsList)
}
如果有些參數不想傳,也可以使用默認值,這樣也不會影響該程序的正常使用
object Hdfs2LocalPath {
// 加載變量
private var hdfsPath: String = _
private var execSQL: String = "select * from temp"
private var outputSchema: String = "csv"
private var headLine: String = "true"
private var writeMode: String = "append"
private var outputPath: String = _
private var numPartitions: Int = 1
}

浙公網安備 33010602011771號