Maarso 19, 2025
Sida Aan Usoo Bandhignay 50 Luuqadood
Markii aan go’aansanay in website-keena ku dhisan GoHugo.io uu noqdo mid luuqado badan leh, waxaan dooneynay hab hufan, ballaaran, isla markaana kharash-ku-ool ah oo lagu abuuro turjumaado. Halkii aan gacanta ku turjumi lahayn bog kasta, waxaan ka faa’iideysanay API-ga OpenAI si aan u otomaatigino habka. Maqaalkaan wuxuu sharxayaa sida aan u dhexgalnay OpenAI API iyo Hugo, annagoo isticmaalayna mawduuca HugoPlate ee Zeon Studio, si aan u soo saarno turjumaado si degdeg ah oo sax ah.
Sababta Aan U Dooranay OpenAI API Turjumaadda
Adeegyada turjumaadda dhaqameed badanaa waxay u baahan yihiin dadaal badan oo gacanta ah, iyo aaladaha otomaatiga sida Google Translate, inkastoo ay waxtar leeyihiin, mararka qaarkood ma bixiyaan heerka habeynta ee aan u baahnayn. API-ga OpenAI wuxuu noo oggolaaday:
- In si ballaaran loo otomaatigiyo turjumaadaha
- In la habeeyo qaabka turjumaadda
- In la ilaaliyo xakameyn wanaagsan oo tayada ah
- In si fudud loogu dhexgalo website-keena ku dhisan Hugo
- In bogag gaar ah loo calaamadiyo dib-u-turjumaad
- In luuqado cusub lagu daro dadaal yar
Habka Talaabo-Talaabo
1. Diyaarinta Website-ka Hugo
Website-keenu wuxuu horey u isticmaalayay mawduuca HugoPlate, kaasoo taageera hawsha luuqado badan. Talaabada koowaad waxay ahayd in la dhaqaajiyo taageerada luuqadaha faylka config/_default/languages.toml ee Hugo:
```toml
################ English language ##################
[en]
languageName = "English"
languageCode = "en-us"
contentDir = "content/english"
weight = 1
################ Arabic language ##################
[ar]
languageName = "العربية"
languageCode = "ar"
contentDir = "content/arabic"
languageDirection = 'rtl'
weight = 2
```
Qaabeyntan waxay hubinaysaa in Hugo uu abuuri karo noocyo luuqado kala duwan leh oo ka mid ah waxyaabaha ku jira.
2. Otomaatigida Turjumaadaha iyadoo la adeegsanayo OpenAI API
Waxaan horumarinay script Bash ah oo otomaatig u turjuma faylasha Markdown. Script-kan:
- Wuxuu akhriyaa faylasha
.mdee Ingiriisiga ah ee ku jira galka ilaha. - Wuxuu adeegsanayaa OpenAI API si uu u turjumo qoraalka iyadoo la ilaalinayo qaabka Markdown.
- Wuxuu qoraa waxyaabaha la turjumay galalka luuqadaha ku habboon.
- Wuxuu la socdaa xaaladda turjumaadda isagoo adeegsanaya fayl JSON ah.
Halkan waxaa ku jira script-ka bash ee otomaatigida turjumaadaha. Si xor ah u isticmaal oo wax ka beddel baahidaada:
```bash
# MIT License
# Copyright (c) 2024 Williston Technical Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#!/bin/bash
# ===========================================
# Hugo Content Translation and Update Script (Sequential Processing & New-Language Cleanup)
# ===========================================
# This script translates Hugo Markdown (.md) files from English to all supported target languages
# sequentially (one file at a time). It updates a JSON status file after processing each file.
# At the end of the run, it checks translation_status.json and removes any language from
# translate_new_language.txt only if every file for that language is marked as "success".
# ===========================================
set -euo pipefail
# --- Simple Logging Function (writes to stderr) ---
log_step() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >&2
}
# --- Environment Setup ---
export PATH="/opt/homebrew/opt/coreutils/libexec/gnubin:$PATH"
# (Removed "Script starting." log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
log_step "SCRIPT_DIR set to: $SCRIPT_DIR"
if [ -f "$SCRIPT_DIR/.env" ]; then
log_step "Loading environment variables from .env"
set -o allexport
source "$SCRIPT_DIR/.env"
set +o allexport
fi
# Load new languages from translate_new_language.txt (if available)
declare -a NEW_LANGUAGES=()
if [ -f "$SCRIPT_DIR/translate_new_language.txt" ]; then
while IFS= read -r line || [[ -n "$line" ]]; do
NEW_LANGUAGES+=("$line")
done <"$SCRIPT_DIR/translate_new_language.txt"
else
log_step "No new languages file found; proceeding with empty NEW_LANGUAGES."
fi
API_KEY="${OPENAI_API_KEY:-}"
if [ -z "$API_KEY" ]; then
log_step "❌ Error: OPENAI_API_KEY environment variable is not set."
exit 1
fi
# Supported Languages (full list)
SUPPORTED_LANGUAGES=("ar" "bg" "bn" "cs" "da" "de" "el" "es" "fa" "fi" "fr" "ha" "he" "hi" "hr" "hu" "id" "ig" "it" "ja" "ko" "ml" "mr" "ms" "nl" "no" "pa" "pl" "pt" "ro" "ru" "sk" "sn" "so" "sr" "sv" "sw" "ta" "te" "th" "tl" "tr" "uk" "vi" "xh" "yo" "zh" "zu")
STATUS_FILE="$SCRIPT_DIR/translation_status.json"
SRC_DIR="$SCRIPT_DIR/Content/english"
log_step "Source directory: $SRC_DIR"
# Check dependencies
for cmd in jq curl; do
if ! command -v "$cmd" >/dev/null 2>&1; then
log_step "❌ Error: '$cmd' is required. Please install it."
exit 1
fi
done
MAX_RETRIES=5
WAIT_TIME=2 # seconds
# Create/initialize status file if missing
if [ ! -f "$STATUS_FILE" ]; then
echo "{}" >"$STATUS_FILE"
log_step "Initialized status file at: $STATUS_FILE"
fi
# --- Locking for Status Updates ---
lock_status() {
local max_wait=10
local start_time
start_time=$(date +%s)
while ! mkdir "$STATUS_FILE.lockdir" 2>/dev/null; do
sleep 0.01
local now
now=$(date +%s)
if ((now - start_time >= max_wait)); then
log_step "WARNING: Lock wait exceeded ${max_wait}s. Forcibly removing stale lock."
rm -rf "$STATUS_FILE.lockdir"
fi
done
}
unlock_status() {
rmdir "$STATUS_FILE.lockdir"
}
update_status() {
local file_path="$1" lang="$2" status="$3"
lock_status
jq --arg file "$file_path" --arg lang "$lang" --arg status "$status" \
'.[$file][$lang] = $status' "$STATUS_FILE" >"$STATUS_FILE.tmp" && mv "$STATUS_FILE.tmp" "$STATUS_FILE"
unlock_status
}
# --- Translation Function ---
translate_text() {
local text="$1" lang="$2"
local retry_count=0
while [ "$retry_count" -lt "$MAX_RETRIES" ]; do
user_message="Translate the following text to $lang. Preserve all formatting exactly as in the original.
$text"
json_payload=$(jq -n \
--arg system "Translate from English to $lang. Preserve original formatting exactly." \
--arg user_message "$user_message" \
'{
"model": "gpt-4o-mini",
"messages": [
{"role": "system", "content": $system},
{"role": "user", "content": $user_message}
],
"temperature": 0.3
}')
response=$(curl -s https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $API_KEY" \
-d "$json_payload")
log_step "📥 Received API response."
local error_type
error_type=$(echo "$response" | jq -r '.error.type // empty')
local error_message
error_message=$(echo "$response" | jq -r '.error.message // empty')
if [ "$error_type" == "insufficient_quota" ]; then
sleep "$WAIT_TIME"
retry_count=$((retry_count + 1))
elif [[ "$error_type" == "rate_limit_reached" || "$error_type" == "server_error" || "$error_type" == "service_unavailable" ]]; then
sleep "$WAIT_TIME"
retry_count=$((retry_count + 1))
elif [ "$error_type" == "invalid_request_error" ]; then
return 1
elif [ -z "$error_type" ]; then
if ! translated_text=$(echo "$response" | jq -r '.choices[0].message.content' 2>/dev/null); then
return 1
fi
if [ "$translated_text" == "null" ] || [ -z "$translated_text" ]; then
return 1
else
translated_text=$(echo "$translated_text" | sed -e 's/^```[[:space:]]*//; s/[[:space:]]*```$//')
echo "$translated_text"
return 0
fi
else
return 1
fi
done
return 1
}
# --- Process a Single File (Sequential Version) ---
process_file() {
local src_file="$1" target_file="$2" lang="$3" rel_src="$4"
# If target file exists and is non-empty, mark status as success.
if [ -s "$target_file" ]; then
update_status "$rel_src" "$lang" "success"
return 0
fi
content=$(<"$src_file")
if [[ "$content" =~ ^[---|\+\+\+]([:space:)]*$ ]] && [[ "$content" =~ [[:space:]]*[---|\+\+\+\+]([:space:)]*$ ]]; then
front_matter=$(echo "$content" | sed -n '/^\(---\|\+\+\+\)$/,/^\(---\|\+\+\+\)$/p')
body_content=$(echo "$content" | sed -n '/^\(---\|\+\+\+\)$/,/^\(---\|\+\+\+\)$/d')
else
front_matter=""
body_content="$content"
fi
log_step "Translating [$rel_src] to $lang..."
translated_body=$(translate_text "$body_content" "$lang")
if [ $? -ne 0 ]; then
update_status "$rel_src" "$lang" "failed"
return 1
fi
mkdir -p "$(dirname "$target_file")"
if [ -n "$front_matter" ]; then
echo -e "$front_matter
$translated_body" >"$target_file"
else
echo -e "$translated_body" >"$target_file"
fi
updated_content=$(echo "$content" | sed -E 's/^retranslate:\s*true/retranslate: false/')
echo "$updated_content" >"$src_file"
update_status "$rel_src" "$lang" "success"
}
# --- Main Sequential Processing ---
ALL_SUCCESS=true
for TARGET_LANG in "${SUPPORTED_LANGUAGES[@]}"; do
log_step "Processing language: $TARGET_LANG"
TARGET_DIR="$SCRIPT_DIR/Content/$TARGET_LANG"
while IFS= read -r -d '' src_file; do
rel_src="${src_file#$SCRIPT_DIR/}"
target_file="$TARGET_DIR/${src_file#$SRC_DIR/}"
# If file is marked not to retranslate, check that target file exists and is non-empty.
if ! [[ " ${NEW_LANGUAGES[@]:-} " =~ " ${TARGET_LANG} " ]] && grep -q '^retranslate:\s*false' "$src_file"; then
if [ -s "$target_file" ]; then
update_status "$rel_src" "$TARGET_LANG" "success"
else
update_status "$rel_src" "$TARGET_LANG" "failed"
fi
continue
fi
process_file "$src_file" "$target_file" "$TARGET_LANG" "$rel_src"
done < <(find "$SRC_DIR" -type f -name "*.md" -print0)
done
log_step "Translation run completed."
end_time=$(date +%s)
duration=$((end_time - $(date +%s)))
log_step "Execution Time: $duration seconds"
if [ "$ALL_SUCCESS" = true ]; then
log_step "🎉 Translation completed successfully for all supported languages!"
else
log_step "⚠️ Translation completed with some errors."
fi
# --- Clean Up Fully Translated New Languages ---
if [ -f "$SCRIPT_DIR/translate_new_language.txt" ]; then
log_step "Cleaning up fully translated new languages..."
for lang in "${NEW_LANGUAGES[@]:-}"; do
incomplete=$(jq --arg lang "$lang" 'to_entries[] | select(.value[$lang] != null and (.value[$lang] != "success")) | .key' "$STATUS_FILE")
if [ -z "$incomplete" ]; then
log_step "All translations for new language '$lang' are marked as success. Removing from translate_new_language.txt."
sed -E -i '' "/^[[:space:]]*$lang[[:space:]]*$/d" "$SCRIPT_DIR/translate_new_language.txt"
else
log_step "Language '$lang' still has incomplete translations."
fi
done
fi
```
3. Maareynta Xaaladda Turjumaadda
Si looga hortago turjumaadyo soo noqnoqda iyo in la la socdo horumarka, waxaan isticmaalnay fayl JSON ah (translation_status.json). Script-ku wuxuu cusbooneysiiyaa faylkan kadib marka dukumeenti kasta la farsameeyo, isagoo hubinaya in kaliya waxyaabaha cusub ama la cusbooneysiiyay la turjumo.
4. Maareynta Khaladaadka iyo Xaddidaadaha Heerka API
Waxaan hirgelinnay isku dayo iyo maareynta khaladaadka si aan ula tacaalno xaddidaadaha heerka, fashilka API, iyo arrimaha quutada. Script-ku wuxuu sugaa ka hor inta uusan mar kale isku dayin haddii OpenAI API uu soo celiyo khalad sida rate_limit_reached ama service_unavailable.
5. Daabacaadda
Marka waxyaabaha la turjumay la soo saaro, orodka hugo --minify wuxuu dhisaa website-ka static-ka ah ee luuqado badan leh, diyaar u ah in la daabaco.
Caqabadaha iyo Xalalka
1. Saxnaanta Turjumaadda
Inkastoo turjumaadaha OpenAI guud ahaan sax ahaayeen, qaar ka mid ah erayada farsamada waxay u baahan karaan dib u eegis gacanta ah, laakiin annagu waxaan nahay koox laba qof ah, sidaas darteed waxaan rajeyneynaa wax wanaagsan. Waxaan hagaajinay prompts si loo ilaaliyo macnaha iyo codka.
2. Dhibaatooyinka Qaabeynta
Mararka qaar qaabka Markdown ayaa isbedel ku yimaada turjumaadda. Si tan loo saxo, waxaan ku darnay habka ka dambeeya ee ilaalinta qaabeynta.
3. Kharashka API-ga oo La Hagaajiyay
Si loo yareeyo kharashka, waxaan hirgelinnay caching si looga fogaado turjumaadyo soo noqnoqda oo aan isbeddelin.
4. Maareynta Dib-u-turjumaadaha Si Hufan
Si aan dib loogu turjumin bogag gaar ah, waxaan ku darnay retranslate: true oo ah parameter hore oo ku jira front matter-ka. Script-ku wuxuu dib u turjumaa kaliya bogagga calaamadeeyay. Tani waxay noo oggolaaneysaa inaan cusbooneysiino turjumaadaha sida loo baahdo annagoo aan dib u turjumin website-ka oo dhan.
Gabagabo
Anagoo dhexgalnay OpenAI API iyo Hugo, waxaan si otomaatig ah u turjunay website-keena annagoo ilaalinayna tayada iyo dabacsanaanta. Habkani wuxuu badbaadiyay waqti, hubiyay isku mid ahaansho, wuxuuna noo oggolaaday inaan si fudud u ballaarinno. Haddii aad dooneyso inaad website-kaaga Hugo ka dhigto mid luuqado badan leh, API-ga OpenAI wuxuu bixiyaa xal awood leh.