---
title: "Introduction to effectcheck"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Introduction to effectcheck}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

## Important notice

> **This package is under active development and has not been fully validated.**
> Results should be independently verified before use in any consequential
> context. Use is at your sole responsibility. Corrections, verification
> reports, and contributions are welcome at
> <https://github.com/giladfeldman/escicheck/issues>.

## Overview

**effectcheck** is a conservative, assumption-aware statistical consistency
checker for research results. It parses test statistics from already-extracted
text, recomputes effect sizes and p-values, and flags discrepancies.

Key design principles:

- **Conservative**: When the study design is ambiguous (paired vs. independent),
  effectcheck computes *all* plausible variants and reports the closest match.
- **Transparent**: Every result includes the assumptions made, the variant
  chosen, and reproducible R code to verify the computation.
- **Comprehensive**: Supports t-tests, F-tests/ANOVA, correlations, chi-square,
  z-tests, regression coefficients, and nonparametric tests (U, W, H).

## Quick start

```{r basic}
library(effectcheck)

# Check a single APA-style result
result <- check_text("t(28) = 2.21, p = .035, d = 0.80")
print(result)
```

The output shows whether the reported effect size is consistent with what
effectcheck recomputes from the test statistic and degrees of freedom.

## Checking multiple statistics

effectcheck handles multiple statistics in the same text:

```{r multiple}
text <- "
Study 1 found a significant effect, t(45) = 3.12, p = .003, d = 0.91.
The ANOVA revealed a main effect, F(2, 87) = 5.44, p = .006.
The correlation was significant, r(48) = .42, p = .003.
"

results <- check_text(text)
summary(results)
```

## Understanding the output

Each row in the results represents one detected statistic. Key columns:

- **test_type**: The type of test (t, F, r, chisq, z, U, W, H, regression)
- **status**: PASS, OK, NOTE, WARN, ERROR or SKIP (see "Status levels" below)
- **effect_reported / matched_value**: the reported effect size and the closest
  same-type computed variant (`matched_variant` names it)
- **delta_effect**: absolute difference between the two
- **uncertainty_level** (`low`, `medium`, `high`) with **uncertainty_reasons** and
  **assumptions_used**: what effectcheck had to assume

Every column is described in `docs/output-columns.md` in the source repository
(<https://github.com/giladfeldman/escicheck/blob/main/docs/output-columns.md>).

## Working with results

effectcheck returns S3 objects with convenient methods:

```{r filter}
# Filter by status
errors <- get_errors(results)
warnings <- get_warnings(results)

# Filter by test type
t_tests <- filter_by_test_type(results, "t")

# Get counts
count_by(results, "test_type")
```

## Examining variants

When effectcheck cannot determine the study design, it computes multiple
variants. You can inspect these:
```{r variants}
result <- check_text("t(28) = 2.21, p = .035, d = 0.80")

# See all computed variants for the first result
format_variants(result, 1)

# Get metadata about a specific variant type
get_variant_metadata("d_ind")
```

## Checking documents

effectcheck checks **text**. Since version 0.4.0 it does not read PDF, DOCX or HTML files;
the former file functions (`check_file()`, `check_files()`, `check_dir()`, `checkPDF()` and
friends) are defunct. Extract the text with a document extractor such as docpluck
(<https://docpluck.app>) and pass it to `check_text()`:

```{r files, eval=FALSE}
paper_text <- paste(readLines("paper.txt", warn = FALSE), collapse = "\n")
result <- check_text(paper_text)
```

## Exporting results

```{r export, eval=FALSE}
results <- check_text("t(28) = 2.21, p = .035, d = 0.80")

# HTML report
generate_report(results, out = file.path(tempdir(), "report.html"))

# CSV/JSON export
export_csv(results, out = file.path(tempdir(), "results.csv"))
export_json(results, out = file.path(tempdir(), "results.json"))
```

## Comparison with statcheck

If the [statcheck](https://CRAN.R-project.org/package=statcheck) package is
installed, you can run both tools side-by-side:

```{r statcheck, eval=FALSE}
comp <- compare_with_statcheck("t(28) = 2.21, p = .035, d = 0.80")
print(comp)
```

## Status levels

Every detected statistic receives one of six statuses:

| Status | Meaning |
|--------|---------|
| **PASS** | Reported effect size matches a same-type computed variant within tolerance (or an APA rounding match) |
| **OK** | The p-value is consistent; no effect size was checked |
| **NOTE** | Could not be fully verified (missing information, CI mismatch, extraction suspect, cross-type comparison) |
| **WARN** | Effect-size distance above the tolerance and up to 5x it, or a downgraded error on an ambiguous design |
| **ERROR** | Effect-size distance above 5x the tolerance |
| **SKIP** | Extracted, but nothing could be checked |

## Supported test types

| Type | Example APA format |
|------|-------------------|
| t-test | `t(28) = 2.21, p = .035` |
| F-test | `F(2, 87) = 5.44, p = .006` |
| Correlation | `r(48) = .42, p = .003` |
| Chi-square | `chi2(1, N = 100) = 4.50, p = .034` |
| z-test | `z = 2.33, p = .020` |
| Mann-Whitney | `U = 145.00, p = .023` |
| Wilcoxon | `W = 89.00, p = .041` |
| Kruskal-Wallis | `H(2) = 8.73, p = .013` |
| Regression | `b = 0.45, SE = 0.12, t(98) = 3.75, p < .001` |