b3/2ticketss
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

# 2ticketss

Browse Source 
2 complementary command-line tools for GitHub issue management
- **00-jira-to-gh-issues**: A Rust tool that converts Jira CSV exports to GitHub issue markdown files compatible with gh-issue-generator. It handles messy CSV data and preserves issue metadata
- **01-gh-issue-generator**: A Rust tool that creates GitHub issues from Markdown files with YAML front matter. It parses structured Markdown, supports batch processing, and integrates with GitHub CLI
This commit is contained in:
bennettldavid
2025-04-04 22:32:49 -06:00
parent 3cf9748684
commit f74bab9ed4
16 changed files with 19626 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
00-jira-to-gh-issues/Cargo.toml Normal file
View File

@@ -0,0 +1,17 @@
[package]
name = "jira-to-gh-issues"
version = "0.1.0"
edition = "2021"
description = "A tool to convert Jira CSV exports to GitHub issue markdown format"
authors = ["b3"]
[dependencies]
clap = { version = "4.4", features = ["derive"] }
csv = "1.2"
anyhow = "1.0"
chrono = "0.4"
serde = { version = "1.0", features = ["derive"] }
dirs = "5.0"
toml = "0.8"
log = "0.4"
env_logger = "0.10"

232
00-jira-to-gh-issues/README.md Normal file
View File

@@ -0,0 +1,232 @@
# Jira to GitHub Issues Converter
A Rust command-line tool that converts Jira CSV exports to GitHub issue markdown files compatible with the [gh-issue-generator](../01-gh-issue-generator) tool.
## Prerequisites
- [Rust](https://www.rust-lang.org/tools/install) installed (cargo, rustc)
- Basic familiarity with Jira CSV exports
## Installation
### Option 1: Build from source
1. Clone this repository
```bash
git clone https://github.com/bee8333/2ticketss.git
cd 2ticketss/00-jira-to-gh-issues
```
2. Build the application:
```bash
cargo build --release
```
3. The executable will be in `target/release/jira-to-gh-issues`
4. Optional: Add to your PATH
```bash
# Linux/macOS
cp target/release/jira-to-gh-issues ~/.local/bin/
# or add the following to your .bashrc or .zshrc
export PATH="$PATH:/path/to/2ticketss/00-jira-to-gh-issues/target/release"
```
### Option 2: Install with Cargo
If you have Rust installed, you can install directly with:
```bash
cargo install --path .
```
## Exporting Issues from Jira
1. In Jira, go to the issue list view (with all filters applied)
2. Click "Export" and select "CSV (All fields)"
3. Save the CSV file
Required Jira fields:
- Summary (required)
- Issue key (required)
Recommended fields (for best results):
- Issue Type
- Priority
- Description
- Assignee
- Reporter
- Created
- Labels
## Usage
```bash
jira-to-gh-issues [OPTIONS] --input <jira_export.csv>
```
Arguments:
- `--input, -i <jira_export.csv>`: Path to the CSV file exported from Jira (required)
Options:
- `--output, -o <output_dir>`: Directory where the markdown files will be saved
- `--status, -s <status>`: Default status for the issues (draft or ready)
- `--verbose, -v`: Increase verbosity level (can be used multiple times: -v, -vv)
- `--config, -c <config>`: Path to config file
### Logging Options
You can control log verbosity in two ways:
1. Using the `--verbose` flag:
- `-v`: Debug level logging
- `-vv`: Trace level logging
2. Using the `RUST_LOG` environment variable:
```bash
# Set global log level
RUST_LOG=debug jira-to-gh-issues --input issues.csv
# Set per-module log level
RUST_LOG=jira_to_gh_issues=trace,csv=warn jira-to-gh-issues --input issues.csv
```
## Configuration
The tool supports configuration files to store default settings. Configuration is searched in these locations (in order):
1. Path specified with `--config` option
2. `.jira-to-gh-issues.toml` in current directory
3. `~/.jira-to-gh-issues.toml` in home directory
4. `~/.config/jira-to-gh-issues/config.toml` in XDG config directory
A default configuration file will be created at `~/.config/jira-to-gh-issues/config.toml` if none exists.
### Configuration Options
```toml
# Default output directory
output_dir = "00-jira-to-gh-issues/output"
# Default issue status (draft or ready)
default_status = "ready"
# Required fields in CSV
required_fields = ["Summary", "Issue key"]
```
### Creating or Editing Configuration
You can manually create or edit the configuration file, or run the tool once to create a default configuration automatically.
For example, to set custom defaults:
```bash
mkdir -p ~/.config/jira-to-gh-issues
cat > ~/.config/jira-to-gh-issues/config.toml << EOF
output_dir = "github-issues/jira-exports"
default_status = "draft"
required_fields = ["Summary", "Issue key", "Description"]
EOF
```
## Example Workflow
### 1. Export issues from Jira
```bash
# Assuming you've exported Jira issues to "sprint-backlog.csv"
```
### 2. Convert to GitHub issue format
```bash
jira-to-gh-issues --input sprint-backlog.csv --output github-issues
```
### 3. Create GitHub issues using gh-issue-generator
```bash
cd ..
gh-issue-generator --repo myuser/myrepo github-issues/20240405123045/batch.md
```
## How It Works
1. The tool reads a CSV file exported from Jira
2. For each issue in the CSV, it:
- Extracts relevant information (title, description, labels, etc.)
- Converts it to the GitHub issue markdown format with YAML front matter
- Writes individual markdown files for each issue
- Creates a batch file containing all issues
3. Output is saved in a timestamped directory to avoid overwriting previous conversions
4. An error log is generated for any issues that couldn't be processed
## Output Format
The tool generates:
1. Individual markdown files for each issue (named after the Jira issue key)
2. A batch file containing all issues in the format expected by gh-issue-generator
3. An error log file listing any processing issues
Each generated file follows the format required by the gh-issue-generator tool:
```markdown
---
title: Issue title
status: ready (or draft)
labels:
- label1
- label2
assignees:
- assignee1
---
# Issue title
## Description
Issue description...
## Metadata
- **Jira Issue**: JIRA-123
- **Created**: Creation date
- **Reporter**: Reporter name
```
## Mapping Rules
| Jira Field | GitHub Issue Field |
|---------------|---------------------------|
| Summary | title |
| Issue Type | Converted to a label |
| Priority | High/Critical → priority label |
| Labels | labels |
| Description | Description section |
| Assignee | assignees |
| Reporter | Listed in metadata |
| Created | Listed in metadata |
| Issue Key | Listed in metadata & filename |
## Troubleshooting
### CSV Import Issues
- Ensure the CSV export from Jira includes the required fields (Summary, Issue key)
- If the import fails with "Missing required columns", verify your Jira export contains these field names
- For malformed CSV files, try opening in a spreadsheet application and re-saving
### Character Encoding
- If you see corrupted characters, ensure your CSV is UTF-8 encoded
- Some Jira instances export in different encodings depending on region settings
### Large Exports
- For very large exports (1000+ issues), consider splitting into multiple files
- Processing large files may require more memory
## Limitations
- Complex Jira markup may not convert perfectly to Markdown
- Attachments from Jira issues are not transferred
- Comments are not included in the conversion
## License
MIT

548
00-jira-to-gh-issues/src/main.rs Normal file
View File

@@ -0,0 +1,548 @@
use std::fs::{self, File};
use std::io::{Write, Read};
use std::path::PathBuf;
use std::collections::HashMap;
use anyhow::{Context, Result, anyhow};
use chrono::Local;
use clap::Parser;
use csv::ReaderBuilder;
use dirs::home_dir;
use log::{info, warn, error, debug, trace, LevelFilter};
use serde::{Deserialize, Serialize};
/// Jira CSV to GitHub Issues Converter
///
/// This tool converts Jira CSV exports to GitHub issue markdown files
/// that are compatible with the gh-issue-generator tool. It handles field
/// mapping, formatting conversions, and generates both individual markdown
/// files and a batch file for bulk creation of GitHub issues.
#[derive(Parser, Debug)]
#[command(
author,
version,
about,
long_about = "A command-line utility that converts Jira CSV exports to GitHub issue markdown files.
It extracts key fields from the Jira export, including summary, description, issue type, priority,
labels, assignees, and other metadata, then converts them to GitHub-compatible format.
The tool creates a timestamped output directory containing:
- Individual markdown files for each issue
- A batch file containing all issues
- An error log tracking any issues encountered during processing
For best results, ensure your Jira export includes at least the Summary and Issue key fields."
)]
struct Args {
/// Input CSV file exported from Jira
///
/// Path to the CSV file exported from Jira containing issues to convert.
/// Export this file from Jira by going to the issue list view,
/// clicking 'Export' and selecting 'CSV (All fields)'.
#[arg(short, long, required = true)]
input: PathBuf,
/// Output directory for generated markdown files
///
/// Directory where the markdown files will be saved. A timestamped
/// subdirectory will be created within this path to avoid overwriting
/// previous conversions.
#[arg(short, long)]
output: Option<PathBuf>,
/// Default status to use (draft or ready)
///
/// Determines whether created issues are marked as draft or ready.
/// Draft issues will not be created when using gh-issue-generator.
#[arg(short, long)]
status: Option<String>,
/// Verbosity level (can be used multiple times)
///
/// Controls the amount of information displayed during execution:
/// -v: Debug level (detailed information for troubleshooting)
/// -vv: Trace level (very verbose, all operations logged)
///
/// You can also use the RUST_LOG environment variable instead.
#[arg(short, long, action = clap::ArgAction::Count)]
verbose: u8,
/// Path to config file
///
/// Path to a TOML configuration file. If not specified, the tool will
/// search for configuration in standard locations:
/// - .jira-to-gh-issues.toml in current directory
/// - ~/.jira-to-gh-issues.toml in home directory
/// - ~/.config/jira-to-gh-issues/config.toml
#[arg(short = 'c', long)]
config: Option<PathBuf>,
}
#[derive(Debug, Serialize, Deserialize)]
struct Config {
/// Default output directory
output_dir: Option<String>,
/// Default issue status (draft or ready)
default_status: Option<String>,
/// Required fields in CSV (comma-separated list)
#[serde(default = "default_required_fields")]
required_fields: Vec<String>,
}
fn default_required_fields() -> Vec<String> {
vec!["Summary".to_string(), "Issue key".to_string()]
}
impl Default for Config {
fn default() -> Self {
Self {
output_dir: Some("00-jira-to-gh-issues/output".to_string()),
default_status: Some("ready".to_string()),
required_fields: default_required_fields(),
}
}
}
impl Config {
/// Load configuration from file or create default if not exists
fn load(config_path: Option<&PathBuf>) -> Result<Self> {
// If path is provided, try to load from there
if let Some(path) = config_path {
debug!("Loading config from specified path: {}", path.display());
return Self::load_from_file(path);
}
// Try to load from default locations
let config_paths = [
// Current directory
PathBuf::from(".jira-to-gh-issues.toml"),
// Home directory
home_dir().map(|h| h.join(".jira-to-gh-issues.toml")).unwrap_or_default(),
// XDG config directory
home_dir().map(|h| h.join(".config/jira-to-gh-issues/config.toml")).unwrap_or_default(),
];
for path in &config_paths {
if path.exists() {
debug!("Found config file at: {}", path.display());
return Self::load_from_file(path);
}
}
// No config file found, use defaults
debug!("No config file found, using defaults");
Ok(Self::default())
}
/// Load configuration from a specific file
fn load_from_file(path: &PathBuf) -> Result<Self> {
let mut file = File::open(path)
.context(format!("Failed to open config file: {}", path.display()))?;
let mut content = String::new();
file.read_to_string(&mut content)
.context("Failed to read config file")?;
toml::from_str(&content)
.context("Failed to parse config file as TOML")
}
/// Save configuration to file
fn save(&self, path: &PathBuf) -> Result<()> {
debug!("Saving config to: {}", path.display());
// Create parent directories if they don't exist
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)?;
}
let content = toml::to_string_pretty(self)
.context("Failed to serialize config to TOML")?;
let mut file = File::create(path)
.context(format!("Failed to create config file: {}", path.display()))?;
file.write_all(content.as_bytes())
.context("Failed to write config file")?;
Ok(())
}
/// Create a default config file if it doesn't exist
fn create_default_if_not_exists() -> Result<()> {
let default_path = home_dir()
.map(|h| h.join(".config/jira-to-gh-issues/config.toml"))
.ok_or_else(|| anyhow!("Failed to determine home directory"))?;
if !default_path.exists() {
debug!("Creating default config file at: {}", default_path.display());
let default_config = Self::default();
default_config.save(&default_path)?;
info!("Created default config file at: {}", default_path.display());
println!("Created default config file at: {}", default_path.display());
}
Ok(())
}
}
struct ProcessingStats {
total_rows: usize,
successful_conversions: usize,
empty_rows: usize,
parsing_errors: usize,
required_field_missing: HashMap<String, usize>,
}
impl ProcessingStats {
fn new() -> Self {
Self {
total_rows: 0,
successful_conversions: 0,
empty_rows: 0,
parsing_errors: 0,
required_field_missing: HashMap::new(),
}
}
fn print_summary(&self) {
println!("\nProcessing Statistics:");
println!(" Total rows in CSV: {}", self.total_rows);
println!(" Successfully converted: {}", self.successful_conversions);
println!(" Empty rows skipped: {}", self.empty_rows);
println!(" Rows with parsing errors: {}", self.parsing_errors);
if !self.required_field_missing.is_empty() {
println!("\n Required fields missing:");
for (field, count) in &self.required_field_missing {
println!(" {}: {} occurrences", field, count);
}
}
}
}
fn main() -> Result<()> {
let args = Args::parse();
// Initialize logger with appropriate level
let log_level = match args.verbose {
0 => LevelFilter::Info,
1 => LevelFilter::Debug,
_ => LevelFilter::Trace,
};
// Initialize the logger, respecting the RUST_LOG env var if set
let mut builder = env_logger::Builder::new();
// If RUST_LOG is set, use that config; otherwise use our default based on -v flags
match std::env::var("RUST_LOG") {
Ok(_) => {
// RUST_LOG is set, initialize with default env_logger behavior
builder.init();
},
Err(_) => {
// RUST_LOG not set, use our verbosity flag
builder
.filter_level(log_level)
.format_timestamp(Some(env_logger::fmt::TimestampPrecision::Seconds))
.init();
// Log a hint about RUST_LOG for users who use -v
if args.verbose > 0 {
debug!("Tip: You can also control log level with the RUST_LOG environment variable");
}
}
};
info!("Starting Jira to GitHub Issues Converter");
// Load configuration
let config = Config::load(args.config.as_ref())?;
debug!("Loaded configuration: {:?}", config);
// Create default config if it doesn't exist (only in normal operation)
if args.verbose == 0 {
if let Err(e) = Config::create_default_if_not_exists() {
warn!("Failed to create default config file: {}", e);
}
}
// Determine output directory (command line takes precedence over config)
let output_dir = match (&args.output, &config.output_dir) {
(Some(path), _) => path.clone(),
(None, Some(dir)) => {
info!("Using output directory from config: {}", dir);
PathBuf::from(dir)
},
(None, None) => {
let default_path = PathBuf::from("00-jira-to-gh-issues/output");
info!("Using default output directory: {}", default_path.display());
default_path
}
};
// Determine issue status (command line takes precedence over config)
let status = match (&args.status, &config.default_status) {
(Some(s), _) => s.clone(),
(None, Some(s)) => {
info!("Using default status from config: {}", s);
s.clone()
},
(None, None) => {
let default_status = "ready".to_string();
info!("Using default status: {}", default_status);
default_status
}
};
debug!("Input file: {}", args.input.display());
debug!("Output directory: {}", output_dir.display());
debug!("Default status: {}", status);
// Create output directory if it doesn't exist
fs::create_dir_all(&output_dir)?;
// Read CSV file
let file = File::open(&args.input)
.with_context(|| format!("Failed to open input file: {}", args.input.display()))?;
let mut rdr = ReaderBuilder::new()
.has_headers(true)
.flexible(true) // Allow for variable number of fields
.from_reader(file);
// Read headers to identify field positions
let headers = rdr.headers()?.clone();
info!("Found {} columns in CSV", headers.len());
// Check for required columns
let required_fields: Vec<&str> = config.required_fields.iter().map(|s| s.as_str()).collect();
validate_headers(&headers, &required_fields)?;
// Prepare timestamp directory for output
let timestamp = Local::now().format("%Y%m%d%H%M%S").to_string();
let batch_dir = output_dir.join(timestamp);
fs::create_dir_all(&batch_dir)?;
info!("Created output directory: {}", batch_dir.display());
// Create batch file for all issues
let mut batch_file = File::create(batch_dir.join("batch.md"))?;
writeln!(batch_file, "# Jira Issues Batch\n")?;
writeln!(batch_file, "Generated from Jira export on {}\n", Local::now().format("%Y-%m-%d %H:%M:%S"))?;
// Process each row in the CSV
let mut stats = ProcessingStats::new();
let mut error_log = File::create(batch_dir.join("error_log.txt"))?;
for (row_idx, result) in rdr.records().enumerate() {
stats.total_rows += 1;
let record = match result {
Ok(r) => r,
Err(e) => {
stats.parsing_errors += 1;
let error_msg = format!("Error reading row {}: {}\n", row_idx + 1, e);
error!("{}", error_msg.trim());
writeln!(error_log, "{}", error_msg)?;
continue;
}
};
// Extract fields safely with fallback to empty string
let summary = get_field_by_name(&record, &headers, "Summary");
let issue_key = get_field_by_name(&record, &headers, "Issue key");
let issue_type = get_field_by_name(&record, &headers, "Issue Type");
let priority = get_field_by_name(&record, &headers, "Priority");
let description = get_field_by_name(&record, &headers, "Description");
let assignee = get_field_by_name(&record, &headers, "Assignee");
let reporter = get_field_by_name(&record, &headers, "Reporter");
let created = get_field_by_name(&record, &headers, "Created");
// Validate required fields
let mut missing_fields = Vec::new();
if summary.is_empty() { missing_fields.push("Summary"); }
if issue_key.is_empty() { missing_fields.push("Issue key"); }
if !missing_fields.is_empty() {
stats.empty_rows += 1;
for field in &missing_fields {
*stats.required_field_missing.entry(field.to_string()).or_insert(0) += 1;
}
let error_msg = format!("Row {}: Missing required fields: {}\n",
row_idx + 1, missing_fields.join(", "));
warn!("{}", error_msg.trim());
writeln!(error_log, "{}", error_msg)?;
continue;
}
trace!("Processing row {}: Issue Key = {}, Summary = {}", row_idx + 1, issue_key, summary);
// Collect all labels from any "Labels" columns
let mut labels = Vec::new();
for (i, header) in headers.iter().enumerate() {
if header == "Labels" && i < record.len() {
let label = record.get(i).unwrap_or("").trim();
if !label.is_empty() {
labels.push(label.to_string());
}
}
}
// Convert to GitHub issue format
let issue_content = convert_to_github_issue(
&summary, &issue_key, &issue_type, &priority,
&labels, &description, &assignee, &reporter,
&created, &status
);
// Generate a filename based on the issue key
// Sanitize issue key to create a valid filename
let safe_issue_key = sanitize_filename(issue_key);
let filename = format!("{}.md", safe_issue_key);
let filepath = batch_dir.join(&filename);
debug!("Writing issue to file: {}", filepath.display());
// Write to individual file
let mut file = File::create(&filepath)
.with_context(|| format!("Failed to create output file: {}", filepath.display()))?;
file.write_all(issue_content.as_bytes())?;
// Also append to batch file
writeln!(batch_file, "---ISSUE---\n")?;
writeln!(batch_file, "{}", issue_content)?;
stats.successful_conversions += 1;
}
// Print summary
stats.print_summary();
info!("Successfully converted {} Jira issues to GitHub issue format", stats.successful_conversions);
println!("\nSuccessfully converted {} Jira issues to GitHub issue format", stats.successful_conversions);
println!("Individual issue files are in: {}", batch_dir.display());
println!("Batch file is at: {}", batch_dir.join("batch.md").display());
println!("Error log is at: {}", batch_dir.join("error_log.txt").display());
Ok(())
}
/// Validate CSV headers for required fields
fn validate_headers(headers: &csv::StringRecord, required_fields: &[&str]) -> Result<()> {
let mut missing = Vec::new();
for &field in required_fields {
if !headers.iter().any(|h| h == field) {
missing.push(field);
}
}
if !missing.is_empty() {
return Err(anyhow!("Missing required columns in CSV: {}. Please ensure your Jira export includes these fields.",
missing.join(", ")));
}
Ok(())
}
/// Sanitize a string to be used as a filename
fn sanitize_filename(s: &str) -> String {
s.chars()
.map(|c| if c.is_alphanumeric() || c == '-' || c == '_' || c == '.' { c } else { '_' })
.collect()
}
/// Safely get a field by column name, handling missing columns
fn get_field_by_name(record: &csv::StringRecord, headers: &csv::StringRecord, name: &str) -> String {
for (i, header) in headers.iter().enumerate() {
if header == name && i < record.len() {
return record.get(i).unwrap_or("").to_string();
}
}
String::new()
}
/// Convert a Jira issue to GitHub issue markdown format
fn convert_to_github_issue(
summary: &str,
issue_key: &str,
issue_type: &str,
priority: &str,
labels: &[String],
description: &str,
assignee: &str,
reporter: &str,
created: &str,
default_status: &str
) -> String {
// Determine appropriate labels based on issue type and priority
let mut gh_labels = Vec::new();
// Add issue type as a label
let issue_type_label = match issue_type {
"Bug" | "Bug⚠️" => "bug",
"Feature" | "Feature🔺" => "enhancement",
"Refactor" | "Refactor♻️" => "refactor",
"Task" | "Task✔️" => "task",
_ => "other",
};
gh_labels.push(issue_type_label.to_string());
// Add priority as a label if significant
if priority.contains("High") || priority.contains("Critical") {
gh_labels.push("high-priority".to_string());
}
// Add any Jira labels
for label in labels {
for l in label.split(',') {
let trimmed = l.trim();
if !trimmed.is_empty() {
gh_labels.push(trimmed.to_string());
}
}
}
// Format labels for YAML front matter
let labels_yaml = if gh_labels.is_empty() {
"".to_string()
} else {
let mut result = "labels:\n".to_string();
for label in gh_labels {
result.push_str(&format!(" - {}\n", label));
}
result
};
// Format assignees for YAML front matter
let assignees_yaml = if assignee.is_empty() {
"".to_string()
} else {
format!("assignees:\n - {}\n", assignee)
};
// Format the description, replacing any Jira formatting with Markdown
let formatted_description = description
.replace("\\n", "\n")
.replace("{noformat}", "```")
.replace("!image-", "![image-");
// Create GitHub issue content with YAML front matter
let content = format!(
"---\ntitle: {}\nstatus: {}\n{}{}---\n\n# {}\n\n## Description\n\n{}\n\n## Metadata\n\n- **Jira Issue**: {}\n- **Created**: {}\n- **Reporter**: {}\n",
summary,
default_status,
labels_yaml,
assignees_yaml,
summary,
formatted_description,
issue_key,
created,
reporter
);
content
}