paramGradientCheck

NeuralNetwork.paramGradientCheck(name, params, delta_params, batch_Y, exp_work, cost, batch_idx, layer_idx, last_delta_y_dt)

パラメータごとの勾配の計算のチェック

引数:

• name (string) – パラメータ名
• params (float[]) – パラメータの配列
• delta_params (float[]) – パラメータのδの配列
• batch_Y (float[]) – 正解の出力
• exp_work (float[]) – 作業用データ
• cost (double) – コスト
• batch_idx (int) – ミニバッチ内のインデックス
• layer_idx (int) – レイヤーのインデックス
• last_delta_y_dt (float[]) – 最後のレイヤーのδy

ソース

paramGradientCheck(name, params, delta_params, batch_Y, exp_work, cost, batch_idx, layer_idx, last_delta_y_dt){
    Assert(params.length == delta_params.length);
    // delta bias
    var idx_list = random.RandomSampling(params.length, 3);

    for(let i of idx_list){
        var b = params[i];
        var db = delta_params[i];
        var eps = b * 0.01;

        params[i] = b - eps;
        var cost1 = this.forwardCost(batch_Y, exp_work, batch_idx, layer_idx, last_delta_y_dt);

        params[i] = b + eps;
        var cost2 = this.forwardCost(batch_Y, exp_work, batch_idx, layer_idx, last_delta_y_dt);

        var diff = db * 2 * eps - (cost2 - cost1);
        console.log("delta-%s : %f dC:%f eps:%f cost:%f,%f,%f", name, diff, db, eps, cost1, cost - cost1, cost2 - cost);

        params[i] = b;
    }
}