The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
The pseudobibeR package calculates the lexicogrammatical and functional features described by Biber (1985) and widely used for text-type, register, and genre classification tasks. The package extracts 67 different linguistic features from pre-parsed text data, enabling researchers to analyze linguistic variation across different text types and registers.
This package doesn’t perform part-of-speech tagging itself. Instead, it leverages existing high-quality taggers from udpipe or spaCy (via spacyr) to extract and aggregate linguistic patterns. The accuracy of feature extraction depends on the quality of the underlying part-of-speech tagging and dependency parsing.
Note: Texts with irregular spellings, non-normative punctuation, or domain-specific language may produce less reliable outputs unless the taggers are specifically tuned for those purposes.
Install the stable version from CRAN:
install.packages("pseudobibeR")Or install the development version from GitHub:
# install.packages("devtools")
devtools::install_github("browndw/pseudobibeR")The package provides the main function biber(), which
takes either udpipe- or spacyr-tagged text and produces a data frame of
linguistic features for each document.
library(spacyr)
library(pseudobibeR)
# Initialize spaCy (requires Python and spaCy installation)
spacy_initialize(model = "en_core_web_sm")
# Parse text and extract features
features <- biber(
spacy_parse(
c("doc_1" = "The task was done by Steve",
"doc_2" = "I think that you should reconsider your approach"),
dependency = TRUE,
tag = TRUE,
pos = TRUE
)
)
print(features)library(udpipe)
library(pseudobibeR)
# Download and load udpipe model
ud_model <- udpipe_download_model(language = "english")
ud_model <- udpipe_load_model(ud_model$file_model)
# Parse text and extract features
text_data <- data.frame(
doc_id = c("doc_1", "doc_2"),
text = c("The task was done by Steve",
"I think that you should reconsider your approach")
)
parsed_data <- udpipe_annotate(ud_model,
x = text_data$text,
doc_id = text_data$doc_id,
tagger = "default",
parser = "default")
features <- biber(parsed_data)
print(features)The package includes sample data to demonstrate functionality:
library(pseudobibeR)
# Extract features from sample spaCy-parsed data
spacy_features <- biber(spacy_samples)
head(spacy_features)
# Extract features from sample udpipe-parsed data
udpipe_features <- biber(udpipe_samples)
head(udpipe_features)Choose one of the following parsing pipelines:
en_core_web_sm)See the respective package documentation for installation instructions.
The biber() function accepts the following
arguments:
tokens: Pre-parsed text data from
spacyr::spacy_parse() or
udpipe::udpipe_annotate()measure: Type-token ratio measure
("MATTR", "TTR", "CTTR",
"MSTTR", or "none")normalize: Whether to normalize counts to per 1,000
tokens (TRUE/FALSE)# Example with custom parameters
features <- biber(parsed_data,
measure = "MATTR",
normalize = TRUE)pseudobibeR uses testthat for comprehensive unit testing. To avoid distributing large language models with the package, tests use pre-saved parsing outputs.
tests/testthat/text-samples/samples.tsv: Sample
sentences for testingtests/testthat/text-samples/parse-samples.R: Script to
generate parsed samplestests/testthat/text-samples/*.rds: Saved parsing
outputs for testsIf you modify samples.tsv, regenerate the parsed
samples:
# Navigate to tests/testthat/text-samples/
source("parse-samples.R")# Run all tests
testthat::test_package("pseudobibeR")
# Run specific test files
testthat::test_file("tests/testthat/test-features.R")The package includes several built-in datasets:
dict: Dictionary patterns for feature detection (e.g.,
third-person pronouns, conjunctions)word_lists: Exact word lists for specific
featuresspacy_samples: Sample spaCy-parsed text dataudpipe_samples: Sample udpipe-parsed text dataThese can be examined to understand feature definitions:
# View available dictionaries
names(dict)
# Examine specific word lists
word_lists$f_06_first_person_pronouns
# Load sample data
data(spacy_samples)
data(udpipe_samples)When using pseudobibeR in your research, please cite:
The original Biber (1985) paper: > Biber, D. (1985). Investigating macroscopic textual variation through multifeature/multidimensional analyses. Linguistics, 23(2), 337-360. DOI: 10.1515/ling.1985.23.2.337
This package: > Brown, D. W. (2024). pseudobibeR: Aggregate Counts of Linguistic Features. R package version 1.1. https://github.com/browndw/pseudobibeR
The package extracts 67 linguistic features organized into 16 categories based on Biber (1985). Each feature is identified through a combination of part-of-speech patterns, dependency relations, and lexical matching.
| Category | Features | Description |
|---|---|---|
| A. Tense and aspect markers | f_01-f_03 | Past tense, perfect aspect, present tense |
| B. Place and time adverbials | f_04-f_05 | Spatial and temporal adverbials |
| C. Pronouns and pro-verbs | f_06-f_12 | Personal pronouns, demonstratives, indefinites, pro-verb do |
| D. Questions | f_13 | Direct wh-questions |
| E. Nominal forms | f_14-f_16 | Nominalizations, gerunds, other nouns |
| F. Passives | f_17-f_18 | Agentless and by-passives |
| G. Stative forms | f_19-f_20 | be as main verb, existential there |
| H. Subordination features | f_21-f_38 | Various subordinate clause types |
| I. Prepositional phrases, adjectives and adverbs | f_39-f_42 | Prepositional phrases, attributive/predicative adjectives, adverbs |
| J. Lexical specificity | f_43-f_44 | Type-token ratio, average word length |
| K. Lexical classes | f_45-f_51 | Conjuncts, downtoners, hedges, amplifiers, emphatics, etc. |
| L. Modals | f_52-f_54 | Possibility, necessity, and predictive modals |
| M. Specialized verb classes | f_55-f_58 | Public, private, suasive verbs, seem/appear |
| N. Reduced forms and dispreferred structures | f_59-f_63 | Contractions, deletions, split constructions |
| O. Co-ordination | f_64-f_65 | Phrasal and clausal coordination |
| P. Negation | f_66-f_67 | Synthetic and analytic negation |
| Feature | Description |
|---|---|
| - | A. Tense and aspect markers |
| f_01_past_tense | Past tense |
| f_02_perfect_aspect | Perfect aspect |
| f_03_present_tense | Present tense |
| - | B. Place and time adverbials |
| f_04_place_adverbials | Place adverbials (e.g., above, beside, outdoors) |
| f_05_time_adverbials | Time adverbials (e.g., early, instantly, soon) |
| - | C. Pronouns and pro-verbs |
| f_06_first_person_pronouns | First-person pronouns |
| f_07_second_person_pronouns | Second-person pronouns |
| f_08_third_person_pronouns | Third-person personal pronouns (excluding it) |
| f_09_pronoun_it | Pronoun it |
| f_10_demonstrative_pronoun | Demonstrative pronouns (that, this, these, those as pronouns) |
| f_11_indefinite_pronoun | Indefinite pronounes (e.g., anybody, nothing, someone) |
| f_12_proverb_do | Pro-verb do |
| - | D. Questions |
| f_13_wh_question | Direct wh-questions |
| - | E. Nominal forms |
| f_14_nominalization | Nominalizations (ending in -tion, -ment, -ness, -ity) |
| f_15_gerunds | Gerunds (participial forms functioning as nouns) |
| f_16_other_nouns | Total other nouns |
| - | F. Passives |
| f_17_agentless_passives | Agentless passives |
| f_18_by_passives | by-passives |
| - | G. Stative forms |
| f_19_be_main_verb | be as main verb |
| f_20_existential_there | Existential there |
| - | H. Subordination features |
| f_21_that_verb_comp | that verb complements (e.g., I said [that he went].) |
| f_22_that_adj_comp | that adjective complements (e.g., I’m glad [that you like it].) |
| f_23_wh_clause | wh-clauses (e.g., I believed [what he told me].) |
| f_24_infinitives | Infinitives |
| f_25_present_participle | Present participial adverbial clauses (e.g., [Stuffing his mouth with cookies], Joe ran out the door.) |
| f_26_past_participle | Past participial adverbial clauses (e.g., [Built in a single week], the house would stand for fifty years.) |
| f_27_past_participle_whiz | Past participial postnominal (reduced relative) clauses (e.g., the solution [produced by this process]) |
| f_28_present_participle_whiz | Present participial postnominal (reduced relative) clauses (e.g., the event [causing this decline[) |
| f_29_that_subj | that relative clauses on subject position (e.g., the dog [that bit me]) |
| f_30_that_obj | that relative clauses on object position (e.g., the dog [that I saw]) |
| f_31_wh_subj | wh- relatives on subject position (e.g., the man [who likes popcorn]) |
| f_32_wh_obj | wh- relatives on object position (e.g., the man [who Sally likes]) |
| f_33_pied_piping | Pied-piping relative clauses (e.g., the manner [in which he was told]) |
| f_34_sentence_relatives | Sentence relatives (e.g., Bob likes fried mangoes, [which is the most disgusting thing I’ve ever heard of].) |
| f_35_because | Causative adverbial subordinator (because) |
| f_36_though | Concessive adverbial subordinators (although, though) |
| f_37_if | Conditional adverbial subordinators (if, unless) |
| f_38_other_adv_sub | Other adverbial subordinators (e.g., since, while, whereas) |
| - | I. Prepositional phrases, adjectives and adverbs |
| f_39_prepositions | Total prepositional phrases |
| f_40_adj_attr | Attributive adjectives (e.g., the [big] horse) |
| f_41_adj_pred | Predicative adjectives (e.g., The horse is [big].) |
| f_42_adverbs | Total adverbs |
| - | J. Lexical specificity |
| f_43_type_token | Type-token ratio (including punctuation) |
| f_44_mean_word_length | Average word length (across tokens, excluding punctuation) |
| - | K. Lexical classes |
| f_45_conjuncts | Conjuncts (e.g., consequently, furthermore, however) |
| f_46_downtoners | Downtoners (e.g., barely, nearly, slightly) |
| f_47_hedges | Hedges (e.g., at about, something like, almost) |
| f_48_amplifiers | Amplifiers (e.g., absolutely, extremely, perfectly) |
| f_49_emphatics | Emphatics (e.g., a lot, for sure, really) |
| f_50_discourse_particles | Discourse particles (e.g., sentence-initial well, now, anyway) |
| f_51_demonstratives | Demonstratives |
| - | L. Modals |
| f_52_modal_possibility | Possibility modals (can, may, might, could) |
| f_53_modal_necessity | Necessity modals (ought, should, must) |
| f_54_modal_predictive | Predictive modals (will, would, shall) |
| - | M. Specialized verb classes |
| f_55_verb_public | Public verbs (e.g., assert, declare, mention) |
| f_56_verb_private | Private verbs (e.g., assume, believe, doubt, know) |
| f_57_verb_suasive | Suasive verbs (e.g., command, insist, propose) |
| f_58_verb_seem | seem and appear |
| - | N. Reduced forms and dispreferred structures |
| f_59_contractions | Contractions |
| f_60_that_deletion | Subordinator that deletion (e.g., I think [he went].) |
| f_61_stranded_preposition | Stranded prepositions (e.g., the candidate that I was thinking [of]) |
| f_62_split_infinitve | Split infinitives (e.g., He wants [to convincingly prove] that …) |
| f_63_split_auxiliary | Split auxiliaries (e.g., They [were apparently shown] to …) |
| - | O. Co-ordination |
| f_64_phrasal_coordination | Phrasal co-ordination (N and N; Adj and Adj; V and V; Adv and Adv) |
| f_65_clausal_coordination | Independent clause co-ordination (clause-initial and) |
| - | P. Negation |
| f_66_neg_synthetic | Synthetic negation (e.g., No answer is good enough for Jones.) |
| f_67_neg_analytic | Analytic negation (e.g., That isn’t good enough.) |
Contributions are welcome! Please feel free to submit issues or pull requests on GitHub.
When reporting bugs or requesting features, please include:
This package is licensed under the MIT License. See the LICENSE file for details.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.