louis/oktopus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #243 from rogersacchelli/dev

Browse Source 
Kubernetes Deployment
This commit is contained in:
Leandro Antônio Farias Machado 2024-04-17 12:41:15 -03:00 committed by GitHub
commit a430857ce3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
20 changed files with 611 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
.github/workflows/frontend.yaml vendored Normal file
View File

@ -0,0 +1,39 @@
name: Oktopus Frontend
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build-frontend:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build Docker image
run: docker build -t ${{ secrets.DOCKERHUB_USERNAME }}/frontend:latest -f frontend/build/Dockerfile frontend
-
name: Push Docker image to Docker Hub
run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/mqtt:latest

39
.github/workflows/mqtt.yaml vendored Normal file
View File

@ -0,0 +1,39 @@
name: Oktopus MQTT Broker
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build-mqtt:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build Docker image
run: docker build -t ${{ secrets.DOCKERHUB_USERNAME }}/mqtt:latest -f backend/services/mtp/mqtt/build/Dockerfile backend/services/mtp/mqtt
-
name: Push Docker image to Docker Hub
run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/mqtt:latest

91
deploy/kubernetes/README.md Normal file
View File

@ -0,0 +1,91 @@
# Oktopus Kubernetes
## Requirements
Kubernetes 1.28+
### Standalone Installation
Single Node:
* 8 vCPUs
* 8 GB RAM
# Installation
## Download Files
```shell
git clone https://github.com/OktopUSP/oktopus
export DEPLOYMENT_PATH=oktopus/deploy/kubernetes
```
## MongoBD
```shell
# Mongo DB Operator at mongodb namespace
helm repo add mongodb https://mongodb.github.io/helm-charts
helm install community-operator mongodb/community-operator --namespace mongodb --create-namespace
# Mongo DB ReplicaSet
export DEPLOYMENT_PATH=oktopus/deploy/kubernetes
kubectl apply -f $DEPLOYMENT_PATH/mongodb.yaml -n mongodb
# Check Installation
kubectl get pods -n mongodb
```
## NATS Server
```shell
# Download the NATS charts
helm repo add nats https://nats-io.github.io/k8s/helm/charts/
# Install NATS with Jetstream Enabled
helm install nats nats/nats --set config.jetstream.enabled=true
```
## Oktopus
<b>Node Ports</b>
For this deployment, we are not using a load balancer and kubernetes is deployed on-premises so we are using Nodeports to insource the client traffic into cluster. below the ports set on deployment files:
1. MQTT broker service (mqtt-svc): 30000
2. Frontend (frontend-svc): 30001
3. SocketIO: (socketio-svc): 30002
4. Controller (controller-svc): 30003
5. WebSocket (ws-svc): 30005
Before deploying the files, edit the frontend.yaml file to set the correct enviroment variables:
```yaml
env:
- name: NEXT_PUBLIC_REST_ENDPOINT
value: "<FRONTEND_IP>:30003"
- name: NEXT_PUBLIC_WS_ENDPOINT
value: "<FRONTEND_IP>:30005"
```
```shell
kubectl apply -f $DEPLOYMENT_PATH/mqtt.yaml
kubectl apply -f $DEPLOYMENT_PATH/mqtt-adapter.yaml
kubectl apply -f $DEPLOYMENT_PATH/adapter.yaml
kubectl apply -f $DEPLOYMENT_PATH/controller.yaml
kubectl apply -f $DEPLOYMENT_PATH/socketio.yaml
kubectl apply -f $DEPLOYMENT_PATH/frontend.yaml
kubectl apply -f $DEPLOYMENT_PATH/ws.yaml
kubectl apply -f $DEPLOYMENT_PATH/ws-adapter.yaml
```
### Checking cluster status:
```shell
kubectl get pods
kubectl get svc
```

39
deploy/kubernetes/adapter.yaml Normal file
View File

@ -0,0 +1,39 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: adapter
spec:
replicas: 1
selector:
matchLabels:
app: adapter
strategy:
type: Recreate # Specify the Recreate strategy
template:
metadata:
labels:
app: adapter
spec:
containers:
- name: adapter
image: oktopusp/adapter:latest
resources:
requests:
memory: 64Mi
cpu: 0.1
limits:
memory: 128Mi
cpu: 0.2
imagePullPolicy: IfNotPresent
env:
- name: NATS_URL
value: "nats://nats:4222"
- name: NATS_NAME
value: "adapter"
- name: NATS_VERIFY_CERTIFICATES
value: "false"
- name: MONGO_URI
value: "mongodb://oktopusp:oktopusp@mongodb-0.mongodb-svc.mongodb.svc.cluster.local:27017,mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017,mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017/adapter?replicaSet=mongodb&ssl=false"
- name: CONTROLLER_ID
value: "oktopusController"

21
deploy/kubernetes/backend.yaml
View File

@ -1,21 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
labels:
app: backend
spec:
replicas: 1
selector:
matchLabels:
app: backend
template:
metadata:
labels:
app: backend
spec:
containers:
- name: backend
image: backend:latest
ports:
- containerPort: 8080

53
deploy/kubernetes/controller.yaml Normal file
View File

@ -0,0 +1,53 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: controller
spec:
replicas: 1
selector:
matchLabels:
app: controller
strategy:
type: Recreate
template:
metadata:
labels:
app: controller
spec:
containers:
- name: controller
image: oktopusp/controller:latest
resources:
# requests:
# memory: 64Mi
# cpu: 0.5
# limits:
# memory: 256Mi
# cpu: 1
imagePullPolicy: IfNotPresent
env:
- name: NATS_URL
value: "nats://nats:4222"
- name: NATS_NAME
value: "adapter"
- name: NATS_VERIFY_CERTIFICATES
value: "false"
- name: MONGO_URI
value: "mongodb://oktopusp:oktopusp@mongodb-0.mongodb-svc.mongodb.svc.cluster.local:27017,mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017,mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017/?replicaSet=mongodb&ssl=false"
- name: REST_API_PORT
value: "8000"
---
apiVersion: v1
kind: Service
metadata:
name: controller-svc
spec:
selector:
app: controller
ports:
- protocol: TCP
port: 8000
targetPort: 8000
nodePort: 30003
type: NodePort

48
deploy/kubernetes/frontend.yaml Normal file
View File

@ -0,0 +1,48 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
replicas: 1
selector:
matchLabels:
app: frontend
strategy:
type: Recreate # Specify the Recreate strategy
template:
metadata:
labels:
app: frontend
spec:
containers:
- name: frontend
image: oktopusp/frontend:latest
resources:
requests:
memory: 64Mi
cpu: 100m
limits:
memory: 256Mi
cpu: 200m
ports:
- containerPort: 3000
imagePullPolicy: IfNotPresent
env:
- name: NEXT_PUBLIC_REST_ENDPOINT
value: "192.168.1.130:30003"
- name: NEXT_PUBLIC_WS_ENDPOINT
value: "192.168.1.130:30005"
---
apiVersion: v1
kind: Service
metadata:
name: frontend-svc
spec:
selector:
app: frontend
ports:
- protocol: TCP
port: 3000
targetPort: 3000
nodePort: 30001
type: NodePort

7
deploy/kubernetes/haproxy-kubernetes-ingress-cm.yaml Normal file
View File

@ -0,0 +1,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: haproxy-kubernetes-ingress
namespace: haproxy-controller
data:
syslog-server: "address:stdout, format: raw, facility:daemon"

8
deploy/kubernetes/haproxy-tcp-services-cm.yaml Normal file
View File

@ -0,0 +1,8 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: haproxy-tcp
namespace: default
data:
1883: # Port where the frontend is going to listen to.
default/mqtt-svc:1883 # Kubernetes service in the format NS/ServiceName:ServicePort

8
deploy/kubernetes/haproxy-tcp-services.yaml Normal file
View File

@ -0,0 +1,8 @@
controller:
service:
tcpPorts:
- name: mqtt
port: 1883
targetPort: 1883
extraArgs:
- --configmap-tcp-services=default/haproxy-tcp

38
deploy/kubernetes/mongodb.yaml Normal file
View File

@ -0,0 +1,38 @@
---
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongodb
spec:
members: 3
type: ReplicaSet
version: "6.0.5"
security:
authentication:
modes: ["SCRAM"]
users:
- name: oktopusp
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: mongo-secret
roles:
- name: clusterAdmin
db: admin
- name: userAdminAnyDatabase
db: admin
- name: readWriteAnyDatabase
db: admin
scramCredentialsSecretName: my-scram
additionalMongodConfig:
storage.wiredTiger.engineConfig.journalCompressor: zlib
# the user credentials will be generated from this secret
# once the credentials are generated, this secret is no longer required
---
apiVersion: v1
kind: Secret
metadata:
name: mongo-secret
type: Opaque
stringData:
password: oktopusp

44
deploy/kubernetes/mqtt-adapter.yaml Normal file
View File

@ -0,0 +1,44 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: mqtt-adapter
spec:
replicas: 1
selector:
matchLabels:
app: mqtt-adapter
template:
metadata:
labels:
app: mqtt-adapter
spec:
containers:
- name: mqtt-adapter
image: oktopusp/mqtt-adapter:latest
resources:
requests:
memory: 64Mi
cpu: 0.1
limits:
memory: 128Mi
cpu: 0.2
imagePullPolicy: IfNotPresent
env:
- name: NATS_URL
value: "nats:4222"
- name: NATS_NAME
value: "mqtt-adapter"
- name: NATS_VERIFY_CERTIFICATES
value: "false"
- name: MQTT_URL
value: "tcp://mqtt:1883"
- name: MQTT_CLIENT_ID
value: "mqtt-adapter"
- name: MQTT_USERNAME
value: ""
- name: MQTT_PASSWORD
value: ""
- name: MQTT_QOS
value: "1"
- name: MQTT_SERVICE_HOST
value: "mqtt"

48
deploy/kubernetes/mqtt.yaml Normal file
View File

@ -0,0 +1,48 @@
apiVersion: v1
kind: Service
metadata:
name: mqtt-svc
spec:
selector:
app: mqtt
ports:
- protocol: TCP
port: 1883
targetPort: 1883
nodePort: 30000
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mqtt
spec:
replicas: 1
selector:
matchLabels:
app: mqtt
template:
metadata:
labels:
app: mqtt
spec:
containers:
- name: mqtt
image: rogersacchelli/mqtt:latest
ports:
- containerPort: 1883
resources:
requests:
memory: 64Mi
cpu: 100m
limits:
memory: 256Mi
cpu: 200m
imagePullPolicy: IfNotPresent
env:
- name: MQTT_PORT
value: ":1883"
- name: MQTT_TLS
value: "false"
- name: LOG_LEVEL
value: "0" # 0 - DEBUG

35
deploy/kubernetes/socketio.yaml Normal file
View File

@ -0,0 +1,35 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: socketio
spec:
replicas: 1
selector:
matchLabels:
app: socketio
template:
metadata:
labels:
app: socketio
spec:
containers:
- name: socketio
image: oktopusp/socketio:latest
imagePullPolicy: IfNotPresent
env:
- name: NATS_URL
value: "nats:4222"
---
apiVersion: v1
kind: Service
metadata:
name: socketio-svc
spec:
selector:
app: socketio
ports:
- protocol: TCP
port: 5000
targetPort: 5000
nodePort: 30002
type: NodePort

46
deploy/kubernetes/ws-adapter.yaml Normal file
View File

@ -0,0 +1,46 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-adapter
spec:
replicas: 1
selector:
matchLabels:
app: ws-adapter
template:
metadata:
labels:
app: ws-adapter
spec:
containers:
- name: ws-adapter
image: oktopusp/ws-adapter:latest
resources:
#requests:
# memory: 64Mi
# cpu: 0.1
#limits:
# memory: 256Mi
# cpu: 0.2
imagePullPolicy: IfNotPresent
env:
- name: NATS_URL
value: "nats://nats:4222"
- name: NATS_NAME
value: "ws-adapter"
- name: NATS_VERIFY_CERTIFICATES
value: "0" # 0 - DEBUG
- name: WS_TOKEN
value: ""
- name: WS_AUTH_ENABLE
value: "false"
- name: WS_ADDR
value: "ws-svc"
- name: WS_PORT
value: ":8080"
- name: WS_ROUTE
value: "/ws/controller"
- name: WS_TLS_ENABLE
value: "false"
- name: WS_SKIP_TLS_VERIFY
value: "false"

52
deploy/kubernetes/ws.yaml Normal file
View File

@ -0,0 +1,52 @@
apiVersion: v1
kind: Service
metadata:
name: ws-svc
spec:
selector:
app: ws
ports:
- protocol: TCP
port: 8080
targetPort: 8080
nodePort: 30005
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws
spec:
replicas: 1
selector:
matchLabels:
app: ws
template:
metadata:
labels:
app: ws
spec:
containers:
- name: ws
image: oktopusp/ws:latest
ports:
- containerPort: 8080
resources:
requests:
memory: 64Mi
cpu: 100m
limits:
memory: 256Mi
cpu: 200m
imagePullPolicy: IfNotPresent
env:
- name: SERVER_PORT
value: ":8080"
- name: SERVER_AUTH_TOKEN
value: ""
- name: SERVER_AUTH_ENABLE
value: "false"
- name: CONTROLLER_EID
value: "oktopusController"
- name: SERVER_TLS_ENABLE
value: "false"

10
frontend/.env
View File

@ -1,17 +1,17 @@
# ----------------------------- Local Environment ---------------------------- #
NEXT_PUBLIC_REST_ENPOINT="http://localhost:8000/api"
NEXT_PUBLIC_WS_ENPOINT="http://localhost:5000/"
NEXT_PUBLIC_REST_ENDPOINT="http://localhost:8000/api"
NEXT_PUBLIC_WS_ENDPOINT="http://localhost:5000/"
# ---------------------------------------------------------------------------- #
# -------------------------- Production Environment -------------------------- #
#NEXT_PUBLIC_REST_ENPOINT="https://demo.oktopus.app.br/api"
#NEXT_PUBLIC_WS_ENPOINT="https://demo.oktopus.app.br/"
#NEXT_PUBLIC_REST_ENDPOINT="https://demo.oktopus.app.br/api"
#NEXT_PUBLIC_WS_ENDPOINT="https://demo.oktopus.app.br/"
# ---------------------------------------------------------------------------- #
# ---------------------------- Mocked Environment ---------------------------- #
#NEXT_PUBLIC_REST_ENPOINT="https://d9962fd9-2464-4a30-9a86-a15a04b57ad0.mock.pstmn.io"
#NEXT_PUBLIC_REST_ENDPOINT="https://d9962fd9-2464-4a30-9a86-a15a04b57ad0.mock.pstmn.io"
# ---------------------------------------------------------------------------- #

4
frontend/build/Dockerfile
View File

@ -2,11 +2,11 @@ FROM node:16.20.2-alpine as builder
WORKDIR /app
COPY ../ .
COPY ../ ./
RUN npm install
RUN NEXT_PUBLIC_REST_ENPOINT=REST_API_URL NEXT_PUBLIC_WS_ENPOINT=WS_URL npm run build
RUN NEXT_PUBLIC_REST_ENDPOINT=REST_API_URL NEXT_PUBLIC_WS_ENDPOINT=WS_URL npm run build
RUN ls -la && echo "Listing directory contents done"

12
frontend/build/entrypoint.sh
View File

@ -3,14 +3,14 @@ set -Ex
function apply_path {
echo "Check that we have NEXT_PUBLIC_REST_ENPOINT vars"
test -n "$NEXT_PUBLIC_REST_ENPOINT"
echo "Check that we have NEXT_PUBLIC_REST_ENDPOINT vars"
test -n "$NEXT_PUBLIC_REST_ENDPOINT"
echo "Check that we have NEXT_PUBLIC_WS_ENPOINT vars"
test -n "$NEXT_PUBLIC_WS_ENPOINT"
echo "Check that we have NEXT_PUBLIC_WS_ENDPOINT vars"
test -n "$NEXT_PUBLIC_WS_ENDPOINT"
find /app/.next \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i "s#REST_API_URL#$NEXT_PUBLIC_REST_ENPOINT#g"
find /app/.next \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i "s#WS_URL#$NEXT_PUBLIC_WS_ENPOINT#g"
find /app/.next \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i "s#REST_API_URL#$NEXT_PUBLIC_REST_ENDPOINT#g"
find /app/.next \( -type d -name .git -prune \) -o -type f -print0 | xargs -0 sed -i "s#WS_URL#$NEXT_PUBLIC_WS_ENDPOINT#g"
}
apply_path