19 Maris, 2025
Yadda Muka Bayar da Harsuna 50
Lokacin da muka fara yin gidan yanar gizonmu na GoHugo.io na harsuna da yawa, muna son hanya mai inganci, mai iya fadada aiki, kuma mai araha don samar da fassara. Maimakon fassara kowace shafi da hannu, mun yi amfani da API na OpenAI don yin aikin ta atomatik. Wannan labarin yana bayanin yadda muka haɗa OpenAI API tare da Hugo, ta amfani da taken HugoPlate daga Zeon Studio, don samar da fassara cikin sauri da daidai.
Dalilin Zaben OpenAI API don Fassara
Ayyukan fassara na gargajiya sau da yawa suna bukatar ƙoƙari mai yawa na hannu, kuma kayan aikin atomatik kamar Google Translate, duk da amfani, ba koyaushe suke ba da matakin daidaitawa da muka bukata ba. API na OpenAI ya ba mu damar:
- Yin fassara da yawa ta atomatik
- Daidaita salon fassarar
- Riƙe mafi kyawun iko akan inganci
- Haɗawa cikin sauƙi da gidan yanar gizonmu na Hugo
- Alamar shafuka guda don sake fassara
- Ƙara sabbin harsuna cikin ƙoƙari kaɗan
Mataki-Mataki Tsari
1. Shirya Gidan Yanar Gizon Hugo
Gidan yanar gizonmu tuni an saita shi ta amfani da taken HugoPlate, wanda ke goyon bayan aikin harsuna da yawa. Mataki na farko shine kunna tallafin harshe a cikin fayil ɗin config/_default/languages.toml na Hugo:
```toml
################ English language ##################
[en]
languageName = "English"
languageCode = "en-us"
contentDir = "content/english"
weight = 1
################ Arabic language ##################
[ar]
languageName = "العربية"
languageCode = "ar"
contentDir = "content/arabic"
languageDirection = 'rtl'
weight = 2
```
Wannan saitin yana tabbatar da cewa Hugo zai iya samar da nau’ikan harsuna daban-daban na abun cikinmu.
2. Yin Fassara ta Atomatik da OpenAI API
Mun haɓaka Bash script don yin fassarar fayilolin Markdown ta atomatik. Wannan script ɗin:
- Yana karanta fayilolin
.mdna Turanci daga babban fayil ɗin tushe. - Yana amfani da OpenAI API don fassara rubutu yayin kiyaye tsarin Markdown.
- Yana rubuta abun da aka fassara zuwa manyan fayilolin harshe masu dacewa.
- Yana lura da matsayin fassara ta amfani da fayil ɗin JSON.
Ga script ɗin bash ɗinmu don yin fassara ta atomatik. Kuna iya amfani da shi da gyara shi don bukatunku:
```bash
# MIT License
# Copyright (c) 2024 Williston Technical Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#!/bin/bash
# ===========================================
# Hugo Content Translation and Update Script (Sequential Processing & New-Language Cleanup)
# ===========================================
# This script translates Hugo Markdown (.md) files from English to all supported target languages
# sequentially (one file at a time). It updates a JSON status file after processing each file.
# At the end of the run, it checks translation_status.json and removes any language from
# translate_new_language.txt only if every file for that language is marked as "success".
# ===========================================
set -euo pipefail
# --- Simple Logging Function (writes to stderr) ---
log_step() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >&2
}
# --- Environment Setup ---
export PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH"
# (Removed "Script starting." log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
log_step "SCRIPT_DIR set to: $SCRIPT_DIR"
if [ -f "$SCRIPT_DIR/.env" ]; then
log_step "Loading environment variables from .env"
set -o allexport
source "$SCRIPT_DIR/.env"
set +o allexport
fi
# Load new languages from translate_new_language.txt (if available)
declare -a NEW_LANGUAGES=()
if [ -f "$SCRIPT_DIR/translate_new_language.txt" ]; then
while IFS= read -r line || [[ -n "$line" ]]; do
NEW_LANGUAGES+=("$line")
done <"$SCRIPT_DIR/translate_new_language.txt"
else
log_step "No new languages file found; proceeding with empty NEW_LANGUAGES."
fi
API_KEY="${OPENAI_API_KEY:-}"
if [ -z "$API_KEY" ]; then
log_step "❌ Error: OPENAI_API_KEY environment variable is not set."
exit 1
fi
# Supported Languages (full list)
SUPPORTED_LANGUAGES=("ar" "bg" "bn" "cs" "da" "de" "el" "es" "fa" "fi" "fr" "ha" "he" "hi" "hr" "hu" "id" "ig" "it" "ja" "ko" "ml" "mr" "ms" "nl" "no" "pa" "pl" "pt" "ro" "ru" "sk" "sn" "so" "sr" "sv" "sw" "ta" "te" "th" "tl" "tr" "uk" "vi" "xh" "yo" "zh" "zu")
STATUS_FILE="$SCRIPT_DIR/translation_status.json"
SRC_DIR="$SCRIPT_DIR/Content/english"
log_step "Source directory: $SRC_DIR"
# Check dependencies
for cmd in jq curl; do
if ! command -v "$cmd" >/dev/null 2>&1; then
log_step "❌ Error: '$cmd' is required. Please install it."
exit 1
fi
done
MAX_RETRIES=5
WAIT_TIME=2 # seconds
# Create/initialize status file if missing
if [ ! -f "$STATUS_FILE" ]; then
echo "{}" >"$STATUS_FILE"
log_step "Initialized status file at: $STATUS_FILE"
fi
# --- Locking for Status Updates ---
lock_status() {
local max_wait=10
local start_time
start_time=$(date +%s)
while ! mkdir "$STATUS_FILE.lockdir" 2>/dev/null; do
sleep 0.01
local now
now=$(date +%s)
if ((now - start_time >= max_wait)); then
log_step "WARNING: Lock wait exceeded ${max_wait}s. Forcibly removing stale lock."
rm -rf "$STATUS_FILE.lockdir"
fi
done
}
unlock_status() {
rmdir "$STATUS_FILE.lockdir"
}
update_status() {
local file_path="$1" lang="$2" status="$3"
lock_status
jq --arg file "$file_path" --arg lang "$lang" --arg status "$status" \
'.[$file][$lang] = $status' "$STATUS_FILE" >"$STATUS_FILE.tmp" && mv "$STATUS_FILE.tmp" "$STATUS_FILE"
unlock_status
}
# --- Translation Function ---
translate_text() {
local text="$1" lang="$2"
local retry_count=0
while [ "$retry_count" -lt "$MAX_RETRIES" ]; do
user_message="Translate the following text to $lang. Preserve all formatting exactly as in the original.
$text"
json_payload=$(jq -n \
--arg system "Translate from English to $lang. Preserve original formatting exactly." \
--arg user_message "$user_message" \
'{
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "content": $system},
{"role": "user", "content": $user_message}
],
"temperature": 0.3
}')
response=$(curl -s https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $API_KEY" \
-d "$json_payload")
log_step "📥 Received API response."
local error_type
error_type=$(echo "$response" | jq -r '.error.type // empty')
local error_message
error_message=$(echo "$response" | jq -r '.error.message // empty')
if [ "$error_type" == "insufficient_quota" ]; then
sleep "$WAIT_TIME"
retry_count=$((retry_count + 1))
elif [[ "$error_type" == "rate_limit_reached" || "$error_type" == "server_error" || "$error_type" == "service_unavailable" ]]; then
sleep "$WAIT_TIME"
retry_count=$((retry_count + 1))
elif [ "$error_type" == "invalid_request_error" ]; then
return 1
elif [ -z "$error_type" ]; then
if ! translated_text=$(echo "$response" | jq -r '.choices[0].message.content' 2>/dev/null); then
return 1
fi
if [ "$translated_text" == "null" ] || [ -z "$translated_text" ]; then
return 1
else
translated_text=$(echo "$translated_text" | sed -e 's/^```[[:space:]]*//; s/[[:space:]]*```$//')
echo "$translated_text"
return 0
fi
else
return 1
fi
done
return 1
}
# --- Process a Single File (Sequential Version) ---
process_file() {
local src_file="$1" target_file="$2" lang="$3" rel_src="$4"
# If target file exists and is non-empty, mark status as success.
if [ -s "$target_file" ]; then
update_status "$rel_src" "$lang" "success"
return 0
fi
content=$(<"$src_file")
if [[ "$content" =~ ^[---|\+\+\+]([:space:)]*$ ]] && [[ "$content" =~ [[:space:]]*[---|\+\+\+\+]([:space:)]*$ ]]; then
front_matter=$(echo "$content" | sed -n '/^\(---\|\+\+\+\)$/,/^\(---\|\+\+\+\)$/p')
body_content=$(echo "$content" | sed -n '/^\(---\|\+\+\+\)$/,/^\(---\|\+\+\+\)$/d')
else
front_matter=""
body_content="$content"
fi
log_step "Translating [$rel_src] to $lang..."
translated_body=$(translate_text "$body_content" "$lang")
if [ $? -ne 0 ]; then
update_status "$rel_src" "$lang" "failed"
return 1
fi
mkdir -p "$(dirname "$target_file")"
if [ -n "$front_matter" ]; then
echo -e "$front_matter
$translated_body" >"$target_file"
else
echo -e "$translated_body" >"$target_file"
fi
updated_content=$(echo "$content" | sed -E 's/^retranslate:\s*true/retranslate: false/')
echo "$updated_content" >"$src_file"
update_status "$rel_src" "$lang" "success"
}
# --- Main Sequential Processing ---
ALL_SUCCESS=true
for TARGET_LANG in "${SUPPORTED_LANGUAGES[@]}"; do
log_step "Processing language: $TARGET_LANG"
TARGET_DIR="$SCRIPT_DIR/Content/$TARGET_LANG"
while IFS= read -r -d '' src_file; do
rel_src="${src_file#$SCRIPT_DIR/}"
target_file="$TARGET_DIR/${src_file#$SRC_DIR/}"
# If file is marked not to retranslate, check that target file exists and is non-empty.
if ! [[ " ${NEW_LANGUAGES[@]:-} " =~ " ${TARGET_LANG} " ]] && grep -q '^retranslate:\s*false' "$src_file"; then
if [ -s "$target_file" ]; then
update_status "$rel_src" "$TARGET_LANG" "success"
else
update_status "$rel_src" "$TARGET_LANG" "failed"
fi
continue
fi
process_file "$src_file" "$target_file" "$TARGET_LANG" "$rel_src"
done < <(find "$SRC_DIR" -type f -name "*.md" -print0)
done
log_step "Translation run completed."
end_time=$(date +%s)
duration=$((end_time - $(date +%s)))
log_step "Execution Time: $duration seconds"
if [ "$ALL_SUCCESS" = true ]; then
log_step "🎉 Translation completed successfully for all supported languages!"
else
log_step "⚠️ Translation completed with some errors."
fi
# --- Clean Up Fully Translated New Languages ---
if [ -f "$SCRIPT_DIR/translate_new_language.txt" ]; then
log_step "Cleaning up fully translated new languages..."
for lang in "${NEW_LANGUAGES[@]:-}"; do
incomplete=$(jq --arg lang "$lang" 'to_entries[] | select(.value[$lang] != null and (.value[$lang] != "success")) | .key' "$STATUS_FILE")
if [ -z "$incomplete" ]; then
log_step "All translations for new language '$lang' are marked as success. Removing from translate_new_language.txt."
sed -E -i '' "/^[[:space:]]*$lang[[:space:]]*$/d" "$SCRIPT_DIR/translate_new_language.txt"
else
log_step "Language '$lang' still has incomplete translations."
fi
done
fi
```
3. Gudanar da Matsayin Fassara
Don hana maimaita fassara da kuma bin diddigin ci gaba, mun yi amfani da fayil ɗin JSON (translation_status.json). Script ɗin yana sabunta wannan fayil ɗin bayan sarrafa kowace takarda, yana tabbatar da cewa kawai sabon ko sabunta abun ciki ne ake fassara.
4. Magance Kurakurai da Iyakokin API
Mun aiwatar da sake gwadawa da magance kurakurai don magance iyakokin sauri, gazawar API, da matsalolin ƙwaƙwalwar ajiya. Script ɗin yana jira kafin sake gwadawa idan OpenAI API ta dawo da kuskure kamar rate_limit_reached ko service_unavailable.
5. Kaddamarwa
Da zarar an samar da abun cikin da aka fassara, gudanar da hugo --minify yana gina gidan yanar gizon tsaye na harsuna da yawa, wanda ya shirya don kaddamarwa.
Kalubale da Maganganu
1. Daidaiton Fassara
Duk da cewa fassarar OpenAI gabaɗaya tana da daidaito, wasu kalmomin fasaha na iya buƙatar dubawa da hannu, amma mu ƙungiya ce ta mutum biyu kawai, don haka muna fatan mafi kyau. Mun daidaita tambayoyi don kiyaye mahallin da yanayi.
2. Matsalolin Tsari
Tsarin Markdown wani lokaci yana canzawa a cikin fassara. Don gyara wannan, mun ƙara dabarun bayan sarrafawa don kiyaye tsarin.
3. Inganta Farashin API
Don rage farashi, mun aiwatar da ajiyar bayanai (caching) don guje wa sake fassara abun ciki da bai canza ba.
4. Gudanar da Sake Fassara Cikin Inganci
Don sake fassara wasu shafuka na musamman, mun ƙara sigar retranslate: true a cikin front matter. Script ɗin yana sake fassara kawai shafukan da aka yi alama da wannan sigar. Wannan yana ba mu damar sabunta fassara yadda ake bukata ba tare da sake fassara dukan gidan yanar gizon ba.
Kammalawa
Ta hanyar haɗa OpenAI API tare da Hugo, mun atomatik fassara gidan yanar gizonmu yayin kiyaye inganci da sassauci. Wannan hanyar ta adana lokaci, ta tabbatar da daidaito, kuma ta ba mu damar fadada aiki cikin sauƙi. Idan kuna son sanya gidan yanar gizon Hugo ɗinku na harsuna da yawa, API na OpenAI yana ba da mafita mai ƙarfi.