使用 NumPy 和 Python 實現梯度下降

Vaibhav Vaibhav 2024年2月15日

使用 Python 實現梯度下降

``````[[1], [2], [3], [4], [5], [6], [7], ...]
``````

``````[1, 4, 9, 16, 25, 36, 49, ...]
``````

Python 程式碼

``````import random
import numpy as np
import matplotlib.pyplot as plt

def linear_regression(inputs, targets, epochs, learning_rate):
"""
A utility function to run linear regression and get weights and bias
"""
costs = []  # A list to store losses at each epoch
values_count = inputs.shape[1]  # Number of values within a single input
size = inputs.shape[0]  # Total number of inputs
weights = np.zeros((values_count, 1))  # Weights
bias = 0  # Bias

for epoch in range(epochs):
# Calculating the predicted values
predicted = np.dot(inputs, weights) + bias
loss = predicted - targets  # Calculating the individual loss for all the inputs
d_weights = np.dot(inputs.T, loss) / (2 * size)  # Calculating gradient
d_bias = np.sum(loss) / (2 * size)  # Calculating gradient
weights = weights - (learning_rate * d_weights)  # Updating the weights
bias = bias - (learning_rate * d_bias)  # Updating the bias
cost = np.sqrt(
np.sum(loss ** 2) / (2 * size)
)  # Root Mean Squared Error Loss or RMSE Loss
costs.append(cost)  # Storing the cost
print(
f"Iteration: {epoch + 1} | Cost/Loss: {cost} | Weight: {weights} | Bias: {bias}"
)

return weights, bias, costs

def plot_test(inputs, targets, weights, bias):
"""
A utility function to test the weights
"""
predicted = np.dot(inputs, weights) + bias
predicted = predicted.astype(int)
plt.plot(
predicted,
[i for i in range(len(predicted))],
color=np.random.random(3),
label="Predictions",
linestyle="None",
marker="x",
)
plt.plot(
targets,
[i for i in range(len(targets))],
color=np.random.random(3),
label="Targets",
linestyle="None",
marker="o",
)
plt.xlabel("Indexes")
plt.ylabel("Values")
plt.title("Predictions VS Targets")
plt.legend()
plt.show()

def rmse(inputs, targets, weights, bias):
"""
A utility function to calculate RMSE or Root Mean Squared Error
"""
predicted = np.dot(inputs, weights) + bias
mse = np.sum((predicted - targets) ** 2) / (2 * inputs.shape[0])
return np.sqrt(mse)

def generate_data(m, n, a, b):
"""
A function to generate training data, training labels, testing data, and testing inputs
"""
x, y, tx, ty = [], [], [], []

for i in range(1, m + 1):
x.append([float(i)])
y.append([float(i) * a + b])

for i in range(n):
tx.append([float(random.randint(1000, 100000))])
ty.append([tx[-1][0] * a + b])

return np.array(x), np.array(y), np.array(tx), np.array(ty)

learning_rate = 0.0001  # Learning rate
epochs = 200000  # Number of epochs
a = 0.5  # y = ax + b
b = 2.0  # y = ax + b
inputs, targets, train_inputs, train_targets = generate_data(300, 50, a, b)
weights, bias, costs = linear_regression(
inputs, targets, epochs, learning_rate
)  # Linear Regression
indexes = [i for i in range(1, epochs + 1)]
plot_test(train_inputs, train_targets, weights, bias)  # Testing
print(f"Weights: {[x[0] for x in weights]}")
print(f"Bias: {bias}")
print(
f"RMSE on training data: {rmse(inputs, targets, weights, bias)}"
)  # RMSE on training data
print(
f"RMSE on testing data: {rmse(train_inputs, train_targets, weights, bias)}"
)  # RMSE on testing data
plt.plot(indexes, costs)
plt.xlabel("Epochs")
plt.ylabel("Overall Cost/Loss")
plt.title(f"Calculated loss over {epochs} epochs")
plt.show()
``````

Python 程式碼的簡要說明

• `linear_regression(inputs, targets, epochs, learning_rate)`：該函式對資料執行線性迴歸，並返回每個時期的模型權重、模型偏差和中間成本或損失
• `plot_test(inputs, targets, weights, bias)`：該函式接受輸入、目標、權重和偏差，並預測輸入的輸出。然後它會繪製一個圖表來顯示模型預測與實際值的接近程度。
• `rmse(輸入、目標、權重、偏差)`：該函式計算並返回某些輸入、權重、偏差和目標或標籤的均方根誤差。
• `generate_data(m, n, a, b)`：該函式為機器學習模型生成樣本資料，使用方程 `y = ax + b` 進行訓練。它生成訓練和測試資料。`m``n` 分別是指生成的訓練和測試樣本的數量。

輸出

Python 程式碼會將每個 epoch 或迭代的模型訓練狀態輸出到控制檯。如下所示。

``````...
Iteration: 199987 | Cost/Loss: 0.05856315870190882 | Weight: [[0.5008289]] | Bias: 1.8339454694938624
Iteration: 199988 | Cost/Loss: 0.05856243033468181 | Weight: [[0.50082889]] | Bias: 1.8339475347628937
Iteration: 199989 | Cost/Loss: 0.05856170197651294 | Weight: [[0.50082888]] | Bias: 1.8339496000062387
Iteration: 199990 | Cost/Loss: 0.058560973627402625 | Weight: [[0.50082887]] | Bias: 1.8339516652238976
Iteration: 199991 | Cost/Loss: 0.05856024528735169 | Weight: [[0.50082886]] | Bias: 1.8339537304158708
Iteration: 199992 | Cost/Loss: 0.05855951695635694 | Weight: [[0.50082885]] | Bias: 1.8339557955821586
Iteration: 199993 | Cost/Loss: 0.05855878863442534 | Weight: [[0.50082884]] | Bias: 1.8339578607227613
Iteration: 199994 | Cost/Loss: 0.05855806032154768 | Weight: [[0.50082883]] | Bias: 1.8339599258376793
...
``````

``````Weights: [0.5008287639956263]
Bias: 1.8339723159878247
RMSE on training data: 0.05855296238504223
RMSE on testing data: 30.609530314187527
``````

Vaibhav is an artificial intelligence and cloud computing stan. He likes to build end-to-end full-stack web and mobile applications. Besides computer science and technology, he loves playing cricket and badminton, going on bike rides, and doodling.