自定义Python节点规范  

1.模板

在   系统运维>数据挖掘配置>引擎设置 中， 下载自定义Python节点包模板或自定义Python算法节点包模板 。

如果自定义的Python脚本涉及算法训练、预测、评估，且希望和产品内置的训练、预测、评估节点兼容使用，则下载自定义Python算法节点包模板；否则下载自定义Python节点包模板。

模板中需要提供两个文件，一个是Python脚本，一个是节点定义，节点定义的文件格式为json。

例如：Python_split.py（Python脚本），Python_split.json（节点定义）。这两文件名字必须一样，同时也要跟节点定义中的name属性值一样。

2.节点定义

文件中必须包含的属性："name", "alias", "configs", "inputs", "type", "path"。

Python_split.json：

{

        "name":"Python_split",

        "alias":"Python拆分",

        "configs":[

                {

                        "name":"datasetRatio",

                        "lable":"数据集占比",

                        "type":"double",

                        "value":0.7

                }

        ],

        "inputs":[

                {

                        "order":0,

                        "type":[

                                "DATASET"

                        ]

                }

        ],

        "outputs":[

                {

                        "order":0,

                        "type":[

                                "DATASET"

                        ]

                },

                {

                        "order":1,

                        "type":[

                                "DATASET"

                        ]

                }

        ],

        "type":"Python",

        "path":"/自定义模块"

}

2.1属性说明

3.Python脚本

·   文件内容不能为空，且首行只能为：# -*- coding: UTF-8 -*-

·   脚本中必须提供execute(inputs=None,configs=None)方法

·   返回必须是包含一个或多个pandas.DataFrame的列表

3.1示例一（不涉及算法的训练、预测、评估）

  展开源码  

# -*- coding: UTF-8 -*-

from sklearn.model_selection import train_test_split

def execute(inputs=None, configs=None):

        input0 = inputs[0]

        df = input0["dataframe"]

        train_ratio = configs['datasetRatio']

        train_df, test_df = train_test_split(df, train_size=train_ratio)

        return [train_df, test_df]

3.2示例二（涉及算法的训练、预测、评估）

  展开源码  

# -*- coding:utf-8 -*-

import numpy as np

import pandas as pd

# Can not change the name and params of method, only can write code in method body

# Param<df_train>: a pandas.DataFrame of need to train dataset

# Param<feature_cols>: a list of feature column name, if not then empty list

# Param<label_col>: a string of label column name, if not then empty string

# Param<configs>: a dict of node param configs, if not then empty dict

# Must return a model object, because predict step will need it

def fit(df_train=None, feature_cols=None, label_col=None, configs=None):

        print("node param q: " + str(configs['q']))

        from sklearn.linear_model import LinearRegression

        x = df_train.loc[:, feature_cols].dropna()

        y = df_train[label_col].values

        model = LinearRegression()

        model.fit(x, y)

        # Must return a model object, because predict method will need it

        return model

# Can not change the name and params of method, only can write code in method body

# Param<model>: a model object of trained by fit step return

# Param<df_test>: a pandas.DataFrame of need to predict

# Param<feature_cols>: a list of feature column name, if not then empty list

# Param<configs>: a dict of node param configs, if not then empty dict

# Must return a predict dataset as pandas.DataFrame, because evaluate step will need it

def predict(model=None, df_test=None, feature_cols=None, configs=None):

        x = df_test.loc[:, feature_cols].dropna()

        y_pred = model.predict(x)

        y_pred2 = y_pred[:, np.newaxis]

        result = np.concatenate((df_test, y_pred2), axis=1)

        predict_df = pd.DataFrame(result).infer_objects()

        columns = df_test.columns.values.tolist()

        # a column name of predict result, you can name it as others.

        columns += ['prediction']

        predict_df.columns = columns

        # Must return a predict dataset as pandas.DataFrame, because evaluate method will need it

        return predict_df

# Can not change the name and params of method, only can write code in method body

# Param<predict_df>: a pandas.DataFrame of predict by predict step return

# Param<label_col>: a string of label column name, if not then empty string

# Param<configs>: a dict of node param configs, if not then empty dict

# Must return a dict object

def evaluate(predict_df=None, label_col=None, configs=None):

        import sklearn.metrics as metrics

        y_true = predict_df[label_col].values

        # use the column name of predict result by predict step named

        y_pred = predict_df['prediction'].values

        mse = metrics.mean_squared_error(y_true, y_pred)

        mae = metrics.mean_absolute_error(y_true, y_pred)

        r2 = metrics.r2_score(y_true, y_pred)

        rmse = np.sqrt(mse)

        mape = np.mean(np.abs((y_pred - y_true) / y_true)) * 100

        result = {

                'mse': mse,

                'mae': mae,

                'rmse': rmse,

                'r2': r2,

                'mape': mape

        }

        # Must return a dict object

        return result    

3.3Python脚本execute方法参数说明